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ABSTRACT 


The  purpose  of  this  study  was  to  determine  the  most 
appropriate  and  effective  manner  in  which  to  provide  feed- 
back to  the  unit  commander  after  a  Marine  Corps  Combat  Readi- 
ness Evaluation  System  (MCCRES) .   To  this  end  research  was 
conducted  into  the  areas  of  management  control  and  evaluation 
theory . 

The  approach  of  the  study  was  multi-disciplinary  with  a 
detailed  emphasis  on  the  economic  question  of  how  better 
distribution  could  be  made  of  the  scarce  resources  of  man- 
power, equipment,  and  training  time  based  on  the  results  of 
MCCRES.   To  answer  the  questions  a  detailed  field  study  was 
conducted  in  which  interviews  were  completed  with  37  Marine 
officers  of  rank  from  Captain  to  Colonel  who  had  involvement 
with  MCCRES  as  key  billet  holders. 

The  results  of  the  study  are  displayed  as  an  eight-step 
feedback  model  which  is  based  on  accepted  theory  in  the  fields 
of  management  control  and  evaluation  theory.   These  results 
are  communicated  to  the  reader  as  a  descriptive  model. 

The  incorporation  of  this  model  as  the  Standard  MCCRES 
Feedback  Procedure  will  enhance  the  value  of  the  results 
to  the  evaluated  unit  and  will  improve  the  understanding  of 
the  resource  allocation  needs  at  all  levels. 
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I.   INTRODUCTION 

The  Military  has  long  recognized  the  need  for  forces  to 
be  prepared  for  combat.   The  United  States  Marine  Corps  has 
historically  been  considered  this  country's  force  in  readi- 
ness.  A  means  to  evaluate  and  measure  readiness,  called  the 
Marine  Corps  Combat  Readiness  Evaluation  System  (MCCRES) ,    was 
implemented  by  the  Service  in  1977.   MCCRES  was  designed  to 
provide  readiness  information  to  all  levels  of  command.   How- 
ever, the  system's  ultimate  value  lies  in  the  communication  of 
information  to  the  unit  evaluated  and  to  the  higher  level  com- 
mands that  can  support  improved  allocation  of  resources  and, 
ultimately,  a  higher  level  of  readiness.   This  research  views 
the  MCCRES  from  an  interdisciplinary  perspective  and  specifically 
looks  at  the  economic  consideration  of  improving  the  allocation 
of  scarce  resources  of  manpower,  equipment  and  training  time. 

A.  PURPOSE 

The  purpose  of  this  thesis  is  to  look  specifically  at  the 
feedback  resulting  from  a  MCCRES  evaluation  and  to  consider 
all  of  the  MCCRES  information  available  to  the  Marine  Corps 
and  to  determine  how  that  information  could  be  used  to  improve 
combat  readiness. 

B .  SCOPE/METHODOLOGY 

Initially  a  detailed  literature  search  was  conducted  into 
the  areas  of  management  control  and  evaluation  theory. 
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Following  the  detailed  literature  search,  the  background  and 
development  of  MCCRES  was  reviewed.   The  MCCRES  is  applied 
throughout  the  Marine  Corps  to  evaluate  ground  and  air  combat, 
as  well  as  combat  service  support  units.   Most  commonly  the 
MCCRES  is  applied  at  the  infantry  battalion  and  aviation 
squadron  level.   In  total  over  50  0  MCCRES ' s  have  been  conduc- 
ted since  implementation  in  19  77.   Therefore,  a  sizeable  body 
of  Marine  Corps  Officers  still  on  active  duty  have  served  in 
key  billets  during  MCCRES  evaluations.   To  capitalize  upon 
their  experience  a  field  study  was  conducted  which  included 
interviews  with  the  initial  developers  of  MCCRES  (both  Marine 
and  civilian  personnel) .   In  addition  key  questions  on  MCCRES 
feedback  were  posed  to  those  who  had  been  involved  at  all  levels 
of  MCCRES  but  with  special  emphasis  on  those  who  had  served  as 
unit  commanders,  senior  evaluators ,  or  division  or  squadron 
MCCRES  officers.   Based  on  this  research  the  answers  to  ques- 
tions posed  by  this  thesis  were  answered. 

C.   ANALYSIS 

The  information  obtained  was  subjected  to  two  levels  of 
analysis.  First  specific  responses  to  the  field  study  were 
analyzed  through  the  application  of  content  analysis.  Next 
an  analysis  of  general  responses  to  the  data  collected  was 
conducted  identifying  MCCRES  elements  which  were  repeatedly 
presented  by  those  interviewed.  Finally,  a  summary  of  unsolic- 
ited comments  were  compiled  to  provide  completeness  to  the 
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study  and  are  suggested  as  areas  for  further  research  on  the 
subject. 

D.   CONCLUSIONS  AND  RESULTS 

The  conclusions  and  results  are  provided  in  the  form  of  a 
feedback  model  consisting  of  the  following  eight  elements: 

1.  Pre-MCCRES  Brief 

2.  Realtime  Feedback 

3.  Evaluator  Worksheets 

4.  Two-way  Debrief 

5.  Computerized  MCCRES  Printout 

6.  Comparative  Percentile  Results 

7.  Follow-up  Report 

8.  Trend  Analysis 

Overall  the  MCCRES  is  viewed  as  a  valuable  and  necessary 
measure  of  unit  readiness.   However,  to  maximize  the  effec- 
tiveness of  MCCRES  timely  distribution  of  results  must  be  made 
to  the  unit  evaluated  and  all  others  directly  in  the  chain  of 
command.   The  timely  distribution  of  results  will  allow  for 
improved  resource  allocation.   Specifically  the  distribution 
of  results  has  a  high  value  for  improving  the  use  of  limited 
training  time  and  the  development  of  doctrine  that  supports 
the  effective  and  efficient  use  of  Marine  Corps  resources. 
Thus  regular  distribution  of  generic  results  should  be  provided 
to  all  training  and  doctrine  commands  throughout  the  Marine 
Corps  to  improve  areas  consistently  identified  as  deficient. 


14 


II.   BACKGROUND 

The  Marine  Corps  is  interested  in  assessing  Unit  Combat 
Readiness.   Readiness  is  generally  viewed  as  the  peacetime 
level  of  preparedness  to  go  into  combat.   The  Marine  Corps 
Combat  Readiness  Evaluation  System  (MCCRES)  is  designed  to 
measure  units '  level  of  readiness  through  simulated  combat 
conditions  as  determined  by  specific  criteria  set  forth  in  the 
form  of  mission  performance  standards,  tasks  and  requirements, 
and  as  judged  by  experienced,  expert  evaluators  [Ref.  1]. 
The  readiness  demonstrated  is  actually  a  complex  set  of  re- 
sults based  on  the  effective  allocation  of  resources  by  the 
service  and  the  unit  commander  [Ref.  1].   Management  control 
encompasses  all  management  actions  taken  by  the  service  and 
the  unit's  commander  to  bring  the  unit  to  peak  efficiency. 
The  evaluation  system  employed  is  a  detailed  effort  to  quan- 
titatively determine  the  success  of  that  resource  allocation. 
This  chapter  explores  the  key  concept  of  readiness  as  it  has 
historically  evolved.   It  discusses  the  basics  of  MCCRES  and 
lays  groundwork  for  the  detailed  research  that  is  directed  at 
answering  key  questions  involving  feedback  of  MCCRES  data  to 
the  unit  commander  for  the  specific  purpose  of  upgrading 
deficiencies. 

A.   MARINE  CORPS  COMBAT  READINESS  EVALUATION  SYSTEM  (J^CCRES). 
Historically  the  Marine  Corps  has  been  called  upon  as  a 
ready  force  to  meet  contingencies  on  short  notice  world-wide. 
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The  specific  Marine  Corps  Missions  are  detailed  in  the  National 
Security  Act  of  194  7.   The  unique  air-ground  combat  team  organi- 
zation in  which  each  unit  ranging  in  size  from  the  Battalion- 
Landing  Team  (BLT)  to  Marine  Amphibious  Force  (MAF)  is  task 
organized  with  Marine  Corps  Air,  Ground,  Combat  Support,  and 
Combat  Service  Support  elements  making  the  organization  uniquely 
suited  to  respond  rapidly  to  U.S.  global  needs.   Recent  inci- 
dents such  as  the  19  79  Iranian  Crisis  and  the  19  82  Lebanese 
Crisis  have  placed  current  emphasis  on  this  historic  role. 
1.   Purpose 

The  operational  capabilities  of  combat  units  describe 
their  abilities  to  function  for  their  intended  purpose.   It 
is,  therefore,  necessary  that  some  means  be  used  to  estimate 
their  ability  to  perform  their  intended  mission  short  of  actual 
combat.   In  19  76  the  Commandant  of  the  Marine  Corps  (CMC)  iden- 
tified the  need  to  develop  an  operational  readiness  evaluation 
program.   In  his  Posture  Statement  to  Congress  for  19  78,  shortly 
before  MCCRES  was  foinnally  introduced.  General  Louis  H.  Wilson, 
Commandant  of  the  Marine  Corps,  made  the  following  statement 
describing  a  need  for: 

...an  improved  readiness  evaluation  system  to,  provide 
a  timely  and  accurate  evaluation  of  the  readiness  of  the 
Fleet  Marine  Forces,  including  reserve  units,  to  accom- 
plish assigned  missions.   [Ref.  1:  p.  I-A-1] 

The  Marine  Corps  began  using  the  system  in  July,  19  78.   Since 
that  time  the  system  has  been  used  to  test  all  Marine  Corps 
Infantry  Units,  Fixed  Wing  Squadrons,  Rotary  Wing  and  Obser- 
vation Squadrons,  Combat  Support  and  Combat  Service  Support 


16 


element,  both  regular  and  reserve,  at  least  once  every  24 

months.   In  his  FY- 84  Posture  Statement  to  Congress,  General 

Robert  H.  Barrow  (the  Commandant  of  the  Marine  Corps)  made 

the  following  comment  about  MCCRES . 

A  key  element  in  the  readiness  equation  continues  to 
be  operational  performance  standards.   Even  before 
these  units  deploy,  we  evaluate  their  capabilities 
to  accomplish  their  assigned  missions  through  the 
rigorous  Marine  Corps  Combat  Readiness  Evaluation  Sys- 
tem.  The  System  objectively  evaluates  the  combat 
readiness  of  all  active  and  reserve  FMF  (Fleet  Marine 
Force)  Units.   [Ref.  2:  p.  47] 

2 .  Theory 

The  goal  of  MCCRES  is  to  test  the  combat  readiness  of 
Marine  Corps  units.   These  tests  have  been  developed  to  evalu- 
ate the  five  types  of  units  mentioned  previously.   There  are 
difficulties  in  applying  a  standard  such  as  MCCRES.   The  real 
question  that  needs  to  be  answered  for  the  organization  is: 
"Can  the  unit  do  the  job?"  [Ref.  3:  p.  IJ   Since  the  real 
answer  can  only  be  obtained  under  actual  combat  conditions, 
we  must  substitute  simulated  combat  for  actual  combat.   Thus 
the  question  that  is  answered  is:   "How  close  is  the  execution 
to  doctrine?"  [Ref.  3:  p.  1]   Thus  for  MCCRES  we  test  adherence 
to  doctrine,  under  simulated  combat  conditions  as  a  proxy  for 
the  real  question. 

3 .  Scoring/Results 

The  guidance  for  MCCRES  is  defined  in  detail  by  Marine 
Corps  Order  (MCO)  3501.2,  Volumes  I-X.   This  provides  details 
on  how  the  MCCRES  is  applied  to  each  type  of  unit.   The  evalua- 
tion is  broken  down  into  ten  CIO)  categories  that  are  mutually 
exclusive  and  collectively  exhaustive. 
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1.  Reporting  to  higher  level  of  coininand. 

2.  Preparing  for  operations 

3.  Communicating  (including  communications  SOP) 

4.  Performing  as  Marines  (discipline,  dispersion, 
camouflage,  concealment,  using  weapons,  and  so  on) 

5.  Delivering  supporting  fire 

6.  Planning  of  operations 

7.  Conforming  to  doctrine 

8.  Executing  Operations 

9.  Providing  combat  service  support  (including  medical 
support) 

10.  Supervising  required  actions  by  individual  Marines. 
Each  category  corresponds  to  a  vital  aspect  of  the  unit's 
performance  [Ref .  3] . 

For  an  Infantry  Battalion  the  evaluation  is  divided 
into  19  Mission  Performance  Standards,  145  Tasks,  and  79  3  Re- 
quirements.  Each  Battalion  is  not  tested  on  all  MPS,  Tasks, 
and  Requirements  [Refs.  1,  4].   The  final  result  is  a  "Combat 
Ready/Not  Combat  Ready"  for  the  unit  evaluated  based  on  the 
overall  judgment  of  the  senior  evaluator.   Results  are  also 

provided  as  percentile  scores  for  the  Section,  MPS,  and  Tasks. 

« 
The  format  for  information  provided  is  as  follows: 

Infantry  Unit 

Section         2.0        Operations  Performed  %  Score 

MPS  2. A        Actions  by  Marines  %  Score 

Task  2.A.1      Discipline  %  Score 

Requirement     2. A. 1.1    Self  Discipline  %  Score* 

*Note:   Only  a  yes/no  is  given  for  a  Requirement,  thus 

100%  =  Yes  and  0%  =  No  for  each  Requirement  Graded 
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For  example,  the  Infantry  Evaluation  is  subdivided 
into  four  Sections:   Section  2. A  identifies  standards  applica- 
ble to  all  evaluations.   It  is  divided  into  three  MPS,  the 
first  of  which  is  Action  by  Marines.   'Actions  by  Marines'  is 
subdivided  into  13  Tasks — the  first,  'Tasks,  Discipline'  is 
subdivided  into  nine  Requirements.   The  first  of  these  require- 
ments is  'Self-Discipline.'   Requirements  are  evaluated  as: 
Yes/No /Not  Applicable.   Section,  MPS,  Tasks,  and  Requirements' 
overall  evaluations  are  each  given  a  weighted  percentile  score 
based  on  the  number  of  Yes's  for  Requirements  under  the  respec- 
tive category. 

4 .   Procedure 

The  MCCRES  is  divided  for  simplicity  into  five  phases. 
Phase  One  requires  determining  the  appropriate  MPS  to  be  evalu- 
ated based  on  Marine  Corps  Order  3501.2,  Vol.  I,  and  the  appro- 
priate volume  that  applies  to  the  unit  being  evaluated — for 
instance,  air  or  combat-service  support.   Phase  Two  consists 
of  briefing  evaluators  [Ref.  5].   Phase  Three  is  the  actual 
evaluation  under  simulated  combat.   During  this  phase  the 
evaluators  actually  make  the  Yes/No  evaluations  of  the  require- 
ments.  They  are  guided  by  detailed  Key  Indicators  (KI)  which 
assist  the  evaluator  in  making  the  appropriate  choice  [Ref.  4], 
Phase  Four  consists  of  compiling  the  evaluation  results  and 
determining  the  percentile  scores  as  well  as  the  "Combat 
Ready/Not  Combat  Ready"  determination  by  the  senior  evaluator. 

Officially,  there  is  no  relationship  between  the  numeri- 
cal score  for  the  Battalion  and  the  Combat  Ready/Not 
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Combat  Ready  rating.   It  is  possible,  for  example, 
that  one  battalion  with  an  overall  numerical  score  of  50 
could  be  judged  Combat  Ready,  while  another  having  a 
numerical  score  of  75  could  be  judged  Not  Combat  Ready. 
[Ref.  6:  p.  40] 

Phase  Five,  the  final  phase,  is  the  debrief  of  the  evaluated 
unit  by  the  evaluators  as  well  as  the  detailed  written  report 
(computer  printout)  which  is  provided  to  the  evaluated  unit. 
Other  reports  are  forwarded  to  Headquarters  Marine  Corps  (CMC)- 
an  initial  message  report  within  10  days  and  the  detailed  re- 
port within  30  working  days  [Ref.  1] 

B.   READINESS 

At  the  center  of  any  Readiness  Evaluation  System  is  under- 
standing what  is  meant  by  the  key  concept  of  "readiness."   The 
inherent  difficulty  in  .describing  this  term  makes  any  evalua- 
tion, measure,  or  model  of  readiness  all  the  more  complex. 
Thus  we  must  explore  the  meaning  of  "readiness." 

1.   Definition 

Usage  of  the  term  "readiness"  has  tended  to  change 

even  at  the  highest  levels  within  the  Department  of  Defense 

(DDD) .   In  FY  19  77,  then  Secretary  of  Defense,  Donald  H. 

Rumsfeld,  used  the  following  definition  of  readiness: 

"Readiness"  is  a  concept  that  integrates  the  diverse 
factors  that  affect  the  ability  to  deploy,  engage,  and 
sustain  effective  combat  forces.   It  starts  with  the 
overall  ability  and  proficiency  of  U.S.  fighting  men... 
An  equally  important  determinant  of  overall  readiness 
is  the  availability,  capability,  and  condition  of  the 
force's  fighting  equipment.   [Ref.  7:  p.  2] 

In  FY  19  7  8  the  same  Secretary  of  Defense  is  quoted 

as  follows : 
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"Readiness"  refers  to  the  capability  to  respond  adequately 
to  diverse  situations  and  to  sustain  that  response  as 
long  as  necessary.   The  "Readiness"  of  Defense  Combat 
Forces  depends  on  a  myriad  of  diverse  and  often  inter- 
related factors,   [Ref.  7:  p.  2] 

In  recent  years  the  concept  has  taken  a  more  stan- 
dardized definition  under  the  current  administration.   How- 
ever, in  reviewing  historical  literature  concerning  readiness, 
it  must  be  remembered  that  it  does  not  have  consistent  meaning 
In  address  to  Congress  on  February  8,  1982,  and  January  31, 
1983,  the  Secretary  of  Defense  Weinberger  used  the  following 
definition: 

Readiness  is  the  ability  of  a  forces,  units,  weapons 
■  systems,  or  equipment  to  deliver  the  outputs  for  which 
they  were  designed  (including  the  ability  to  deploy  and 
employ  without  unacceptable  delays) .   It  depends  on 
having  the  required  quantities  of  equipment  in  the  hands 
of  the  units  on  a  day-to-day  basis  and  on  having  the 
required  number  of  adequately  trained  people  assigned 
with  the  necessary  mix  of  grades  and  experience  level  and 
to  ensure  that  people  and  machines  can  work  together. 
[Ref.  8:  p.  1-28] 

This  definition  includes  training  (individual  and  unit) , 
material,  equipment,  logistics,  and  personnel  all  as  part  of 
the  readiness  definition. 

For  the  purpose  of  continuity  and  brevity,  the  follow- 
ing definition  of  readiness  is  used  on  a  day-to-day  basis  by 
the  Marine  Corps  and  is  the  accepted  definition  from  JCS , 
Publication  6,  used  for  the  DoD  Unit  Reporting  System 
(UNITREP) . 

Readiness:   ability  of  forces,  units,  weapons  systems, 
or  equipments  to  deliver  outputs  for  which  they  were 
designed  (includes  the  ability  to  deploy  and  employ 
without  unacceptable  delays.   [Ref.  9:  p.  1] 
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It  is  further  amplified  as  follows: 

Readiness  is  essentially  a  measure  of  pre-D-Day  status 
(extending  at  most  into  initial  combat  operations) 
while  s us t a inability  is  a  Post-D-Day  measure.   Hence, 
we  often  speak  of  peace  time  readiness,  but  combat 
sustainability .   [Ref .  9l  pT  T] 

For  consistency  this  definition  of  readiness,  as  amplified, 
will  be  used  throughout  this  study,  since  it  provides  the 
currently  accepted  DoD  definition.   In  the  literature  the 
terms  'effectiveness'  and  'sustainability'  are  often  inter- 
changed to  indicate  Post-D-Day  combat  capabilities.   Because 
of  its  broader  implications  the  term  effectiveness  will  be 
used  herein  to  mean  Post-D-Day  status.   Hereafter  we  can  refer 
to  Pre-Combat  Readiness  and  Combat  Effectiveness . 
2 .   Indexing 

The  difficulty  in  defining  readiness  may  be  exceeded 
only  by  the  difficulty  of  measuring  it.   Many  attempts  have 
been  made  to  index  readiness,  but  to  date  none  have  been  suc- 
cessful or  achieved  wide  acceptance.   As  described  by  Barzily, 
Marlow,  and  Zacks,  in  their  "Survey  of  Approaches  to  Readiness," 
the  motivating  factor  common  to  all  attempts  is  to  measure 
the  effects  of  budgetary  changes  on  readiness.   They  reviewed 
a  number  of  readiness  indexes,  such  as  the  U.S.  Navy  METRI 
Project,  MARIS  Project,  MAXCAP  Models,  and  others  and  found 
that: 

The  readiness  indexes  posed  do  not  attain  the  desired 
objective.   They  are  usually  very  insensitive  to  changes 
that  occur  at  the  lower  echelons  and,  furthermore,  they 
are  generallv  improper  indexes  of  readiness.   [Ref.  10: 
p.  25] 
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3 .   Economic  Question 

The  context  of  readiness  described  in  both  Definitions 

and  Indexing  has  strong  economic  overtones .   The  approach 

herein  will  be  to  treat  readiness  as  an  economic  question, 

i.e.,  the  allocation  of  scarce  resources  of  training  time, 

material,  equipment,  logistics,  and  personnel  as  inputs  with 

the  desired  results  of  maximizing  outputs.   The  true  output  of 

a  combat  organization  is  effectiveness  in  combat.   There  is 

obvious  difficulty  in  measuring  that  output.   Because  of  this 

difficulty  we  generally  turn  our  attention  to  proxy  measures 

such  as  number  of  aircraft  or  number  of  ships.   The  difficulty 

of  this  problem  was  recently  stated  as  follows  in  a  study  by 

Rand  for  the  Assistant  Secretary  of  Defense/Manpower,  Reserve 

Affairs,  and  Logistics: 

The  output  of  an  Army  Maneuver/Firepower  Unit  is 
especially  difficult  because  such  units  have  no  single 
output  or  product  that  can  be  directly  related  to 
their  mission.   [Ref.  12:  p.  7] 

In  a  recent  study  by  Steven  L.  Funk,  several  observa- 
tions were  made  which  have  application  here.   He  states  that 
although  peacetime  Army  readiness  is  an  honored  theme,  it  is 
not  always  practical.   This  is  a  result  of  preparedness  not 
being  the  actual  primary  peacetime  mission,  but  just  one  of 
several  competing  missions  that  take  resources  of  manpower, 
equipment,  and  training  time.   Funk  cites  major  factors  that 
detract  from  the  unit  being  able  to  train  for  combat  situations 

(1)   Lack  of  resources  to  accomplish  requirements 
(i.e.,  skilled  personnel,  time,  ammunition, 
repair  parts,  and  facilities). 
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(2)  changing  and  competing  priorities  that  fragment 
resources  and  negate  planning,  and 

(3)  a  climate  that  deprives  unit  leadership  of 
decision  "maneuver  space"  and  fragments  their 
attention  (i.e.,  centralized  decision  making/ 
control,  burdensome  administration,  low  unit 
cohesion  and  ambiguous  institutional  values. 
[Ref.  11:  p.  6] 

Although  the  previous  study  was  made  of  the  U.S.  Army, 
many  of  the  detractors  apply  equally  to  the  Marine  Corps. 
Some  policies  such  as  Unit  Rotation  (i.e.,  the  policy  of  main- 
taining unit  integrity  by  replacing  an  entire  battalion/ 
squadron  rather  than  replacing  individuals),  and  Deployment- 
Lock-On  periods  (i.e.,  the  policy  of  assigning  resources  to 
a  unit  well  in  advance  of  a  planned  deployment  and  then  main- 
taining and  replacing  those  resources  through  the  training 
cycle  and  into  deployment)  have  lessened  the  negative  effects. 
However,  in  the  Marine  Corps  the  problem  of  competing  priori- 
ties will  always  exist.   The  unit  commander  is  constrained 
by  many  elements  beyond  his  control  [Ref.  11] .   Thus  the  ques- 
tion of  unit  readiness  is  certainly  one  of  economic  priorities 
or  competing  priorities.   A  large  part  of  the  readiness  results 
then  depends  on  the  effective  resource  allocation  both  to  the 
unit  and  within  the  unit.   Thus  MCCRES  attempts  to  evaluate 
the  readiness  of  a  unit  through  simulated  combat  conditions 
and  to  measure  how  closely  the  unit  performance  conforms  to 
doctrine. 

C.   MCCRES  AS  THE  EVALUATION  SYSTEM 
1 .   Research 

The  evaluation  system  employed  by  the  Marine  Corps 
has  undergone  much  study  from  the  standpoint  of  developing  an 


objective  versus  a  subjective  system  of  evaluation.   Much  of 
the  statistical  work  for  this  has  been  done  by  the  George 
Washington  School  of  Engineering  and  Applied  Science  [Refs.  3, 
4].   Additionally,  the  question  of  evaluator  bias  was  recently 
studied  at  the  Naval  Postgraduate  School,  Department  of  Adminis- 
trative Science  [Ref .  5] . 

As  a  result  of  the  George  Washington  University  studies, 
it  was  determined  MCCRES  had  the  following  merits: 

a.  Most  requirements  are  requests  for  descriptive  data 
and  not  for  judgments.   Judgments  were  previously 
made  by  defining  the  requirements  and  assigning 
weights. 

b.  The  details  of  the  doctrine  are  given  and  interpreted 
in  the  requirements,  thus  avoiding  the  possibility 

of  being  mis-interpreted  or  forgotten  by  the 
evaluators. 

c.  The  execution  of  most  requirements  consumes  sKort 
time  periods  and  thus  the  evaluators '  memories  are 
not  overburdened. 

d.  Assigning  a  score  of  a  YES  or  NO  is  easier  than 
assigning  scores  on  any  other  scale. 

e.  The  set  of  the  requirements  exhausts  the  details 
of  the  executions.   [Ref.  3:  p.  25J 

Though  the  George  Washington  University  research  identifies 
merits  of  MCCRES,  it  does  not  address  the  question  of  what 
feedback  should  be  provided  to  the  evaluated  unit. 
2.   Feedback 

The  purpose  of  this  research  was  to  determine  the  types 
of  feedback  that  are  most  useful  and  valuable  to  an  evaluated 
unit.   Feedback  is  a  necessary  part  of  any  control  or  evalua- 
tion system.   In  December,  19  81,  B.  General  A.  A.  Sordo,  then 
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Director  of  the  Marine  Corps  Training  Division,  made  the 

following  comment: 

The  evaluation  and  feedback  process  in  training  is 
one  where  we  need  to  do  substantial  work.   MCCRES 
helps  us  greatly.   In  the  future,  as  we  link  the 
individual  training  standards  of  our  ITS  System  to 
MCCRES  Mission  Performance  Standards,  we'll  be  able 
to  do  better!   But  above  and  beyond  that,  our 
measurements  of  adequacy  and  effectiveness  are  for 
the  most  part  decentralized  and  highly  subjective 
and  impressionistic.   In  the  long  run  we  rely  on 
our  assessment  of  the  unit  or  organizational  capabili- 
ties as  the  best  gauge  of  adequacy  in  training 
preparation.   [Ref.  13:  p.  54] 

D.   RESEARCH  QUESTIONS 

As  a  result  of  the  need  to  define  specifically  the  feed- 
back and  information  needed  by  evaluated  units  as  the  result 
of  MCCRES,  this  study  was  undertaken.   The  following  questions 
are  asked  by  this  thesi,s: 

1.  Primary 

a.  After  a  MCCRES,  what  information  should  be  fed 
back  to  the  evaluated  unit  commander? 

b.  In  what  form  should  the  evaluation  feedback  be? 

c.  What  channel (s)  should  be  used  to  provide  the 
feedback  information  to  the  unit  commander? 

2 .  Secondary 

In  addition  to  the  specific  questions  above,  there  are 
several  subsidiary  questions  which  arise  as  a  logical  conse- 
quence of  the  primary  questions.   They  involve  the  following 
topics:   (1)  validity  of  comparing  MCCRES  results  given  under 
differing  conditions,  (2)  MCCRES  contribution  to  effective 
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training,  and  (3)  time  availability  and  followup  to  ensure 
correction  of  deficiencies  identified  during  the  MCCRES . 

E.  METHODOLOGY  OF  LITERATURE  RESEARCH 

To  properly  answer  the  questions  posed,  the  author  took 
a  multi-disciplinary  approach.   The  theories  of  management 
control  and  evaluation  provide  a  great  deal  of  insight  toward 
understanding  the  complexities  of  the  information  needs.   Thus 
these  two  fields  will  be  explored  in  detail  in  the  next  chapter 
In  addition  much  work  has  been  done  in  the  area  of  modeling 
effectiveness.   This  has  been  done  from  the  perspective  of 
applied,  conceptual,  statistical  and  economic  models. 
Since  these  provide  rich  theories  for  the  understanding  of 
the  measurement  and  indexing  of  effectiveness,  these  concepts 
are  explored. 

F .  SUMMARY 

This  chapter  was  designed  to  provide  a  history  of  MCCRES, 
establish  the  need  for  such  a  system  from  the  Marine  Corps 
perspective,  give  some  basic  understanding  of  the  system,  and 
provide  a  detailed  understanding  of  the  inherent  difficulties 
of  efforts  to  evaluate  the  concept  of  readiness. 

Readiness  is  approached  from  the  standpoint  of  economics, 
that  is  the  application  of  scarce  resources  (inputs)  of  time, 
men,  materials  to  the  process  of  simulated  combat.   The  unit 
commander  controls  the  organization  with  a  highly  sophisticated 
management  control  system  in  accordance  with  the  external  and 
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internal  objectives  and  policies.   MCCRES  is  a  unit  evaluation 
system  which  evaluates  the  management  control  process  of  that 
organization.   The  primary  questions  are  then  presented  as 
the  basis  of  this  research.   The  following  chapters  will  ex- 
plore briefly  the  theory  of  management  control  systems  and 
in  detail  evaluation  theories  and  models  with  specific  empha- 
sis on  feedback  mechanisms. 
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III.   MANAGEMENT,  MANAGEMENT  CONTROL,  EVALUATION, 

AND  FEEDBACK 

The  topics  to  be  covered  in  this  chapter  are  as  follows: 
first  the  topic  of  management  will  be  described  in  a  general 
context;  management  control  and  its  implications  for  feedback 
will  be  explored;  then,  a  series  of  selected  evaluation 
theories  will  be  presented  in  detail.   Finally  a  series  of 
evaluation  models  with  application  to  the  military  will  be 
provided.   Emphasis  in  these  models  will  be  on  feedback 
theory  and  the  ideas  on  feedback  posed  by  the  authors. 
A  summary  of  feedback  as  developed  by  the  models  will  be  pre- 
sented also. 

A .   I^ANAGEMENT 

Management  is  described  as:   (l)"art  or  act  of  managing; 
conduct;  control;  direction.   (2)   Judicious  use  of  means  to 
accomplish  an  end;  skillful  treatment"  [Ref .  14j .   Webber 
describes  it  in  many  contexts--experience,  training,  practice, 
theory,  art,  and  science  [Ref.  15].   The  simplest  definition 
and  one  often  heard  is  the  art  of  getting  things  done  through 
people.   The  elements,  however,  that  all  descriptions  of 
management  have  in  common  are  that  managers  engage  in  two 
important  activities:   planning  and  control  [Ref.  16j .   It  is 
important  to  clearly  make  the  distinction  between  the  terms 
management  and  leadership.   The  military  has  long  recognized 
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that  the  term  "leader"  has  much  broader  implications  than  that 
of  manager.   The  unit  is  more  than  the  sum  of  its  parts;  and, 
therefore,  the  military  commander's  responsibility  is  not  only 
for  the  technical  and  tactical  proficiency  of  the  unit,  but 
for  the  overall  cohesive  functioning  of  the  organization  [Ref, 
17] .   In  this  context  it  is  fair  to  say  that  management  skills 
are  a  necessary  subset  of  leadership  and  that  the  good  leader 
is  necessarily  a  good  manager,  however,  the  converse  of  that 
statement  is  not  necessarily  true. 

B.   MANAGEMENT  CONTROL 

Robert  Anthony  placed  management  control  in  a  hierarchical 
framework  as  follows  in  Figure  3.1. 


Figure  3.1.   Anthony  Framework 

He  described  this  classification  for  any  large  organization 
where  these  functions  are  present. 
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strategic  planning  is  the  process  of  deciding  on 
objectives  of  the  organization,  on  changes  in  the 
objectives,  on  resources  used  to  attain  these 
objectives,  and  on  the  policies  that  are  to  govern 
the  acquisition,  use,  and  disposition  of  these 
resources.   [Ref.  18:  p.  16] 

He  further  described  management  control  as  follows: 

Management  control  is  the  process  by  which  managers 
assure  that  resources  are  obtained  and  used  effec- 
tively and  efficiently  in  the  accomplishment  of  the 
organization's  objectives.   [Ref.  18:  p.  17] 

Strategic  planning  is  the  highest  level  of  management 
where  the  long,  range  objectives,  policies,  and  plans  of  the 
organization  are  made .   The  management  control  level  involves 
middle  management,  the  timeframe  is  shorter,  about  12  months, 
and  the  planning  involves  budgeting,  training,  etc.   The  lowest 
level  of  control  is  operational  where  the  day-to-day  function- 
ing of  the  organization  is  carried  on;  the  implementation  of 
plans  and  policies  are  executed  at  this  level. 

Lebas  describes  management  control  as  an  enlarged  feedback 
loop  which  affects  human  behavior  and  performance  [Ref.  19 J . 
Hofstede  also  provides  a  useful  definition  of  management  con- 
trol "...as  a  pragmatic  concern  for  results,  obtained  through 
people"  [Ref.  24:  p.  193]. 

Given  the  purpose  of  this  study  is  to  identify  feedbaclc 
needed  for  improved  resource  allocation,  it  is  important  to 
focus  on  the  terms  "effectiveness"  and  "efficiency",  which  are 
present  in  many  of  the  definitions  of  management  control.   In 
this  context  the  terms  effectiveness  and  efficiency  are  defined 
as  follows: 
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Eff ectiveness .. .relates  to  the  accomplishment  of  the 
cooperative  purpose .. .when  a  specific  desired  end 
is  attained  we  shall  say  that  the  action  is  effective. 
[Ref.  18:  p.  27] 

Ef ficiency. . .refers  to  the  engineering  sense  of... 
the  optimum  relationship  between  given  inputs  and 
outputs.   [Ref.  18:  p.  27] 

In  Lebas •  work  he  identified  the  major  schools  of  thought 

on  management  control. 

1 .  Organizational  Process  Approach 

The  Organizational  Process  Approach  is  that  taken  by 
Anthony.   He  divides  the  feedback  loop  into  three  distinct 
parts,  described  previously  as  strategic  planning  and  manage- 
ment control  and  operational  control.   This  approach  reduces 
the  human  motivational  problem  to  what  Anthony  calls  "goal 
congruence"  or  the  various  ways  in  which  the  manager  can  be 
encouraged  to  take  actions  which  are  in  the  best  interest  of 
the  company.   The  goal  congruence  is  then  reduced  to  procedures 
within  the  organization  such  as  performance  evaluation,  report- 
ing, and  budgeting  [Ref.  19]. 

2 .  Informational  Economics  Approach 

The  Informational  Economics  Approach  is  a  more  abstract 
theory.   It  proposes  that  only  models  based  on  expected  utility 
maximization  need  to  be  considered.   A  set  of  subjective 
probabilities  are  assumed  to  be  known  by  the  decision-maker 
and  that  the  decision-maker  will  always  act  to  maximize  utility, 
In  this  approach,  good  performance  is  described  as  the  action 
which  minimizes  the  end  result  differences  between  the  expected 
outcomes  and  the  actual  outcomes.   The  emphasis,  however,  is 
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on  the  decision  model  not  the  outcome.   A  further  aspect  of 
the  theory  is  its  emphasis  on  maximizing  the  expected  utility. 
This  leads  to  the  conclusion  that  the  control  process  is  a 
prior  phenomenon  that  suggests  management  spend  more  time 
preparing  decisions  than  evaluating  outcomes  [Ref .  19] . 

The  major  limitation  to  the  Information  Economics  Ap- 
proach is  that  it  is  all  based  on  the  Savage-Rational  Man 
Model  and  that  an  effort  will  be  made  to  maximize  utility  in 
every  case.   Although  an  interesting  theoretical  model,  it 
relies  heavily  on  the  calculus  of  maximizing  utility  in  each 
and  every  case  [Ref.  19]. 

3.   Behavioral  Approach 

This  is  actually  a  loosely  structured  set  of  approaches 

to  the  concept  of  management  control  taking  ideas  from  many 

theories.   Two  descriptive  ideas  that  it  entails  from  Herbert 

Simon  are: 

Satisf icing ...  interrupting  the  action  selection  process 
as  soon  as  the  first  acceptable  action  has  been  found. 

And 

Bounded  Reality ...  the  notion  that  man  is  a  piecewise 
rational.   [Ref.  19] 

Other  ideas  presented  by  Ouchi  and  referred  to  by 

Lebas  have  shown  that: 

...rigorous  output  control  is  the  most  effective  way 
to  induce  good  performance  in  managers.   [Ref.  19] 

Other  topics  in  the  behavioral  approach  deal  with  the 

structural  and  inter-personal  variables  in  management  control. 

The  power,  therefore,  of  each  structural  center  lies  in  its 
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ability  to  reduce  uncertainty  for  other  units.   The  inter- 
personal effects  on  behavior  are  viewed  by  Lebas  in  terms  of 
Kantz  and  Kahn ' s  model  of  role  sender  and  role  receiver.   Here 
the  control  will  be  good  if  the  expectations  are  interpreted 
accurately  by  the  receiver  and  if  the  single  receiver  is  not 
subjected  to  conflicts  in  role  expectations  in  the  different 
social  systems  in  which  it  operates.   Additionally,  control 
requires  acceptance  of  the  role  by  the  receiver  [Ref.  19J . 

Lebas  views  the  real  value  of  behavioral  approaches 
in  their  diversity  and  being  based  on  observations  of  people 
in  real  situations.   The  limitations  come  from  their  multi- 
plicity and  that  creates  difficulty  in  setting  any  sort  of 
coherent  guidelines  for  a  manager  to  employ  [Ref.  19J . 
4 .   Integrated  Approach 

The  previous  broad  structures  of  management  control 
system  have  been  integrated  by  Ansari  in  what  he  refers  to 
as  an  operational  systems  concept  [Ref.  20] .   He  views  the 
previous  perspectives  on  management  control  systems  in  two 
versus  three  broad  categories.   Grouping  the  structural  view 
(Operational  Process  and  Informational  Economics)  and  the  be- 
havioral views: 

a.   Two  Perspectives 

Ansari  describes  the  structural  view  of  management 
control  as  that  adapted  by  researchers  in  cybernetics,  account- 
ing, and  management  information  systems — those  that  concentrate 
primarily  on  the  information  and  communications  aspects  of 
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control  systems.   The  behavioral  approaches  are  identified  as 
those  that  are  based  primarily  on  human  behavior  in  organi- 
zations and  regard  control  as  a  problem  of  encouraging  sub- 
ordinates to  achieve  performance  goals.   In  this  context  he 
uses  control  system  to  describe: 

...those  arrangements  and  actions  designated  to 
facilitate  its  members  to  achieve  higher  performance 
with  least  unintended  consequences.   [Ref.  20:  p.  102] 

b.  Elements  of  Management  Control  Systems 

There  is  agreement  among  most  authors  reviewed 
from  both  the  structural  and  behavioral  schools  that  all 
management  control  systems  consist  of  two  elements.   The 
first  being  "an  information  network  which  prescribes  the  rules 
for  measurement ,  collection,  processing,  and  transmission  of 
information"  [Ref.  20:  p.  102].   This  element  causes  the 
information  on  performance,  goals,  outputs,  and  exceptions 
from  plans  to  be  transmitted  to  managers.   The  second  element 
is  the  set  of  social  relationships  through  which  the  control 
system  achieves  the  organizational  goals  [Ref.  20J . 

c.  Joint  Consideration 

Ansari  argues  that  management  control  is  actually 
best  approached  from  a  combination  of  the  structural  and  be- 
havioral ideas.   He  declares  that  the  current  phase  of  design 
is  more  situational  and  that  it  focuses  on  the  more  important 
issue  of  improving  performance  instead  of  the  narrow  concept 
of  constraining  behavior.   The  information  that  is  provided 
passes  through  two  phases.   First,  it  must  be  perceived  before 
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it  can  be  used  and  second  not  only  managers,  but  subordinates, 
must  also  use  information  for  self-evaluation  and  guidance  of 
performance.   He  further  identifies  five  considerations  which 
are  characteristic  of  all  information  structures. 

1.  Nature  of  Measures — the  subordinates'  motivation 
is  affected  by  the  completeness,  objectivity, 
and  influence  of  the  characteristics  of  perfor- 
mance measures. 

2.  Source  and  Order  of  Presentation — The  source  of 
the  information  must  be  credible  and  the  order 
of  presentation  can  change  the  perceptions  of 
the  information. 

3.  Timing — Both  the  speed  and  frequency  are  impor- 
tant, too  long  an  interval  may  cause  the  user 
to  lose  interest  or  be  distracted. 

4.  Route — The  route  may  alter  the  infoirmation  and 
thus  the  sender  and  receiver  may  not  be  sharing 
the  same  information. 

5.  Shared  Information — The  others  with  whom  the 
information  is  shared  thus  may  affect  the  recipi- 
ent's view  of  the  information  with  regard  to 
fairness  and  accuracy.   [Ref.  20J 

All  of  these  considerations  must  be  made  by  the 
designer  in  any  information,  evaluation  system.   To  summarize 
the  characteristics  of  the  integrated  model,  this  takes  into 
account  both  the  structure  of  the  organization  and  its  infor- 
mational systems  as  well  as  the  social  side  considering  subor- 
dinate personality  and  leadership  style.   The  designer  of  a 
management  control  system  should  combine  components  (such  as 
an  evaluation  system)  in  such  a  way  as  to  minimize  cognitive 
conflicts  and  encourage  behavior  which  resolves  conflict  with 
positive  results  for  the  organization.   It  should  also  be  kept 
in  mind  that  if  rewards  for  performance  are  too  highly 
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contingent  on  outputs  of  the  information  structure,  the  greatej 
the  chance  that  there  will  be  controversy  over  the  output 
[Ref.  20],   A  good  example  of  this  controversy  is  discussed 
by  Anthony  Hopwood  in  the  way  in  which  accounting  data  was 
used  in  the  performance  evaluation  of  managers. 

Based  upon  a  study  he  conducted,  Hopwood  identi- 
fied four  ways  in  which  accounting  data  was  used  in  evaluation, 

(1)  Budget  Constrained  Style — required  to  meet  a  budget 
but  not  concerned  about  costs. 

(2)  Budget  Profit  Style — concerned  with  both  meeting  a 
budget  and  with  costs. 

(3)  Profit  Conscious  Style — concerned  with  costs  but  not 
with  meeting  a  budget. 

(4)  Non-Accounting  Style — not  concerned  with  meeting  a 
budget  or  with  costs.   [Ref.  21] 

He  concluded  that  based  on  management  style,  many  dysfunc- 
tional behaviors  may  occur  as  the  result  of  emphasis  on 
accounting  data  results  in  evaluations  [Ref.  21] . 
5.   Other  Thoughts  on  Management  Control 

Management  control  contains  many  rich  concepts  which 
play  a  great  part  in  the  theory  of  evaluations.   San  Miguel 
has  made  several  interesting  observations  on  management 
control: 

The  end  is  a  system  that  enable  managers  to  make  sound 
decisions  as  to  the  efficient  and  effective  allocation 
of  human,  physical,  and  financial  resources  to  attain 
the  objectives  of  the  organization.   [Ref.  22:  p.  177J 
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San  Miguel  further  characterizes  management  science's 
attempts  to  deal  with  modeling  of  complex  control  systems. 
He  characterizes  the  quantitative  and  economic  decision  tools' 
attempt  to  simplify  the  decision  making  in  large  scale  organi- 
zations as  falling  short  of  their  objectives — primarily  be- 
cause of  the  lack  of  knowledge  of  human  behavior  and  the 
ability  to  quantify  it.   He  further  cites  measurement  and 
communications  to  be  at  the  center  of  control  systems,  internal 
planning,  and  reporting.   There  is  general  agreement  in  the 
management  control  literature  that  measurement  and  communica- 
tions systems  have  an  important  impact  on  the  behavior  of 
individuals  in  organizations,  their  motivations  and,  thus, 
their  performance.   Therefore,  the  areas  of  measurement  and 
evaluation  are  legitimate  concerns  for  the  design  of  any 
management  control  system  [Ref.  22]. 
6 .   Summary 

The  intent  in  this  section  was  to  set  the  stage  through 
the  discussion  of  the  broader  topics,  management  and  manage- 
ment control,  for  what  is  to  be  developed  in  the  next  section 
on  evaluation.   Management  control  is  a  complex  topic  which 
encompasses  the  broad  aspects  of  all  the  control  systems  im- 
posed on  the  organization  through  strategic  plans  and  policies 
as  well  as  those  developed  within  to  assist  the  manager  in 
his  control  internally. 

C.   EVALUATION  OVERVIEW 

Evaluation  models  that  are  directed  at  readiness  all 
embody  a  common  goal .   Each  in  its  own  particular  way  attempts 
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to  identify  a  definitive  measure  or  some  particular  relative 
measure  of  the  unit's  Pre-D-Day  preparedness  for  combat. 
Preparedness  may  be  viewed  as  readiness  which  acts  as  a  proxy 
measure  for  the  military  unit's  ability  to  perform  in  combat. 
Similar  efforts  are  made  to  evaluate  the  ability  to  perform 
in  civilian  organizations.   In  its  simplest  form,  output  is 
measured  by  production  level  or  quantity  produced  by  a  produc- 
tion facility  in  a  given  time  period.   Various  attempts  are 
also  made  to  measure  profitability  of  the  firm.   Microeconomic 
theory  assumes  that  each  firm  is  attempting  to  maximize  profit 
[Ref.  23] .   Therefore,  some  measure(s)  are  applied  to  the 
fiscal  results  of  each  period  to  determine  the  financial  per- 
formance of  the  organization  during  the  period  [Ref.  23J  . 

# 

Similarly  it  can  be  assumed  that  each  unit  tries  to  maximize 
its  readiness.   This  section  reviews  efforts  that  have  been 
made  to  evaluate,  measure,  and  model  performance  both  in 
theory  and  practice.   This  section  covers  the  general  theory 
of  evaluation,  and  looks  at  several  well-known  evaluation 
systems. 

1.   Evaluation  Defined 

The  term  evaluation  is  defined  differently  by  various 
authors;  however.  Stuff lebeam  brings  together  three  generally 
accepted  definitions  of  evaluation  and  provides  a  comparison, 
a.   Measurement  Definition 

Evaluation  is  identical  to  measurement.   It  builds 
on  attempts  to  measure  psychological  attributes  or  character- 
istics [Ref.  24:  pp.  9-10] . 

39 


b.  Congruence  Definition 

Evaluation  is  a  comparison  of  the  congruence 
between  performance  and  objectives.   This  provides  emphasis 
on  objective  performance  also  [Ref.  24]. 

The  measurement  definition  considered  evaluation 
to  occur  after  the  fact  and  to  measure  some  attribute.   In 
contrast  the  congruence  definition  refers  to  on-going  evalua- 
tion in  meeting  the  objectives  [Ref.  27] .   According  to 
Stufflebeam  organizations  will  pick  objectives  which  are 
specific  and  have  objective  measureable  results.   That  is  be- 
cause to  maJce  the  congruence  definition  worlc,  there  must  be 
some  selection  of  objectives  to  be  measured.   Stufflebeam 
further  indicates  that  the  congruence  definition  creates  a 
need  for  the  evaluator  to  find  short  term  measures  of  per- 
formance.  These  indicators,  identifiable  outputs,  or  behaviors 
come  to  be  viewed  as  performance.   These  identifiable  outputs 
become  the  ultimate  criteria  of  all  organizational  decisions. 
The  congruence  definition  has  advantages,  but  the  need  to 
find  measureable  objectives  creates  problems  [Ref.  24J . 

c.  Judgment  Definition 

The  professional  judgment  definition  allows  full 
evaluation  of  all  organizational  attributes  both  quantifiable 
and  non-quantifiable,  and  is  easy  to  implement  iRef.  24]. 
This  definition  is  often  applied  where  the  dimensions  to  be 
measured  are  difficult  to  define  and  more  difficult  to  quan- 
tify.  The  judgment  definition  has  its  problems.   The  evaluation 
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is, by  definition,  based  upon  the  judgment  of  an  individual 
which  lacks  objectivity  identifying  what  data  are  used  and 
how  they  are  used  [Ref.  24]. 

Table  (3.1),  taken  from  Stufflebeam,  provides  a 
comparison  of  advantages  and  disadvantages  of  each  definition. 

2 .  Principles  of  Evaluation 

There  is  a  good  deal  of  general  agreement  that  for 
an  evaluation  to  be  effective  it  must  be  consistent  with  the 
purposes,  objectives,  and  goals  of  the  activity  being 
evaluated  [Refs.  5,25,26]. 

a.  Evaluations  must  be  conducted  in  terms  of  purpose — 
the  evaluator  and  the  evaluated  must  fully  participate 
and  work  for  the  common  goal . 

b.  Evaluation  must  be  cooperative — all  involved  as  both 
the  evaluators  and  evaluated  must  fully  participate 
and  work  for  the  common  goal . 

c.  Evaluation  must  be  continuous — it  must  be  on-going; 
a  one  time  effort  with  no  followup  is  an  affront  to 
the  professional  concepts . 

d.  Evaluation  must  be  specific — specificity  is  the  key, 
generalizations  do  little  good  to  help  remedy  defi- 
ciencies or  to  identify  true  strengths. 

e.  Evaluation  must  provide  the  means  and  focus  for 
trainers  to  be  able  to  appraise  themselves,  their 
practices,  and  their  products. 

f.  Evaluation  must  be  on  a  uniform  and  objective  methods 
and  standards. 

3.  Summary 

The  definition  and  principles  of  evaluation  were  pre- 
sented in  this  section.   The  evaluation  definition  as  described 
by  Stufflebeam  has  three  distinct  interpretations.   There  is 
more  general  agreement  on  the  principles  of  evaluation  and 
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TABLE    3.1 

Advantages   and   Disadvantages   Accruing   from   Different 
Definitions   of   Evaluation 


Advantages 


Disadvantages 


(1)     Measurement 


(2)     Congruence 


(3)  Judgment 


Builds  directly  on  scien-  Narrow  instrumental  focus 
tif  ic  raeasiirement 


Objective 

Reliable 

Data  mathematically 
manipulatable 

Norms  and  standards 
anerge 


High  degree  of  integra- 
tion with  the  process 

Data  available  on 
process  and  structure 

Possibility  of  feedback 


Easy  to  implement 

Brings  all  variables 
into  consideration 

Takes  euq^erieice  and 
expertise  into  accoxjnt 

No  time  lag  v\^ile  wait- 
ing for  data  analysis 


Inflexible  because  of  time 
and  cost  to  produce  new 
instruments 

Obscures  judgments  and  the 
criteria  for  making  them 

Eliminates  variables  cur- 
rently considered  as  not 
measurable  or  labels  than 
unimportant 

Places  evaluator  in  technical 
role 

Focuses  narrowly  on  objectives 

Elevates  behavior  as  the 
ultimate  criterion  of  every 
action 

Focuses  on  evaluation  as  a 
terminal  process 

Dictated  mainly  because 
of  ignorance  or  lack  of 
sophistication 

Questionable  reliability 

Questionable  objectivity 

Not  susceptible  to  ordinary 
scientific,  prudential 
measures 

Both  data  and  criteria  are 
ambiguous 

Generalization  very  difficult 


Source:   Adopted  from  Ref .  24:  p.  15. 
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to  a  large  degree  these  embody  a  common  sense  approach  for  the 
usefulness  of  any  evaluation.   Next,  four  evaluation  theories 
will  be  presented. 

D.   EVALUATION  THEORY  ' 

There  are  many  theories  of  evaluation.   There  is  no  attempt 
herein  to  be  all  inclusive.   What  is  intended  is  to  introduce 
a  variety  of  theories  which  exist  and  to  show  their  relation- 
ship to  the  questions  pursued  in  this  research. 

1.   Management  by  Objective  (MBO) 

In  recent  years  probably  no  system  of  personnel  manage- 
ment has  gained  more  attention  than  Management  by  Objective 
introduced  by  Peter  Drucker  [Ref.  27].   The  MBO  approach  is 
applied  to  individual  as  well  as  organizational  evaluation  sys- 
tems . 

The  MBO  Approach  assumes  that  one  would  perform  more 
effectively  because  you  have  planned  your  own  objectives 
and  could  control  your  own  behavior.   [Ref.  15:  p.  316] 

MBO  takes  a  humanistic  or  human  values  approach  to 

managing  people.   The  basic  steps  in  MBO  are  as  follows: 

1.  The  subordinate  proposes  goals  for  the  next  time 
period. 

2.  The  subordinate  and  superior  discuss,  modify,  and 
reach  agreement. 

3.  Periodic  formal  and  frequent  informal  review. 

4.  Subordinate  reports  on  performance  at  the  end  of 
the  period. 

5.  Repeat  the  cycle.   [Ref.  15:  p.  316] 

MBO  deals  well  with  a  number  of  problem  areas  in  evaluation 
systems.   It  is  designed  to  create  more  two-way  communication. 
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It  has  found  success  in  both  the  private  and  public  sector. 
In  their  article  on  "Employee  Growth  Through  Performance 
Management,"  Beer  and  Ruhr  found  success  with  employment  of 
MBO  at  Corning  Glass  Co.  [Ref.  28].   In  addition,  the  Depart- 
ment of  Health,  Education,  and  Welfare  found  it  a  most  valuable 
system  in  pursuing  management  and  control  of  their  many  diver- 
sified programs  [Ref.  29].   Figure  3.2  provides  a  simple  model 
of  how  MBO  is  designed.   It  should  be  noted  that  the  MBO  model 
focuses  on  performance  as  identified  and  measured  through  ob- 
jectives agreed  upon  by  supervisor  and  subordinate  [Ref.  30]. 

Feedbac]<:  in  the  MBO  System  is  scheduled  at  regular 
intervals.   The  meetings  can  be  used  to  compare  results  to 
the  objectives  set  as  well  as  to  help  make  future  objectives 
more  realistic.   The  objectives  themselves  are  determined  by 
higher  level  management,  but  in  coordination  with  the  employee 
or  unit.   To  a  great  extent  the  comparison  of  goals  set  and 
achieved  are  reported  by  the  individual  being  evaluated. 
Typically  the  manager  submits  an  evaluation  of  each  objective 
based  on  the  following  classification  system: 

1.  Satisfactory 

2.  Minor  Problem 

3.  Major  Problem 

Feedback  is  handled  by  face  to  face  conferences  between  the 
evaluator  and  employee. 

In  a  recent  Marine  Corps  Gazette  article.  Major  James 
M.  Clarke  reflected  his  good  experiences  with  the  MBO  merit 
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Source:   Adapted  from  [Ref.  30:  p.  22  3] 


Figure  3.2.   Sample  MBO  Model 
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pay  system  used  with  high  level  GS  employees.   He  referred 
to  MBO  in  the  context  of  a  personnel  evaluation  system;  how- 
ever, it  also  has  application  here: 

To  improve  performance  counseling,  we  must  get  away 
from  subjective  judgments  through  a  more  formal  system 
which  concentrates  on  objectives.   The  merit  pay 
system  provides  a  good  model.   [Ref.  31:  p.  47] 

Major  Clarke's  article  provided  support  for  the  iyiBO 
method  because  clear  objectives  are  established  and  regular 
communications  are  encouraged.   In  the  Marine  Corps'  personnel 
system,  as  in  other  areas  of  inspection  and  evaluation,  pro- 
fessional judgment  has  historically  been  used  as  the  primary 
method  of  evaluation  and  lack  of  specific  objectives  has  rein- 
forced the  reliance  on  the  judgmental  method. 

MBO  places  a  great  emphasis  on  the  objectives  and  the 
regular  feedback  on  how  the  evaluated  worker  is  measuring  up 
to  those  objectives  [Ref.  15] .   MBO  further  emphasizes  dis- 
playing the  individual's  strengths  and  weaknesses  to  himself, 
rather  than  to  others.   Although  MBO  emphasizes  timely,  accu- 
rate, and  objective  feedback,  the  details  of  that  feedback 
can  only  be  spelled  out  when  a  system  is  input  to  a  specific 
organization  [Ref.  28] .   It  further  encourages  the  feedback 
be  given  on  a  continual  basis,  not  "saved  up  for  an  end  of 
the  year  inquisition"  [Ref.  15:  p.  317]. 
2.   Evaluation  and  Authority 

A  comprehensive  and  ongoing  study  of  evaluation  and 
authority  was  conducted  by  Scott  and  Dornbusch  (19  77) .   In 
their  study  of  professional  organizations,  results  were 
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gathered  from  university  faculty  members,  clergy,  public  school 

teachers,  principals,  nurses,  and  others.   They  found: 

Organizations  are  power  structures  in  which  some 
participants  give  differential  access  to  organizational 
rewards  and  penalties  in  order  to  control  other 
participants.   [Ref.  32:  p.  134] 

The  Scott-Dornbusch  studies  present  a  conceptual  model 

of  the  relation  of  evaluation  and  authority.   As  a  result  of 

these  studies,  they  developed  the  model  presented  in  Figure 

3.3. 
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Source:      Adapted   from    [Ref.    32:    p.    20] 


Figure    3.3.      Model    of   Evaluation  Process  with   Communica- 
tion Links   Among   Right-Holders 


The  model    is   a   systems   analysis   approach  which   provides    for 
monitoring   and  regulating  by  means   of  a   feedback   loop.      This 
approach   concentrates   on  the   involvement  of   each   actor    in   the 
process.      The   actors    in   the   Scott-Dornbusch  model   are   described 
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as  follows:   The  allocator  gives  the  responsibility  for  per- 
forming the  task.   The  criteria  setter  is  concerned  with 
assessing  the  effectiveness  and  efficiency  of  a  task  per- 
formance.  To  a  large  measure  the  criteria  is  determined  by 
the  goal  toward  which  the  performance  is  directed.   The 
determination  of  which  information  to  use  in  order  to  arrive 
at  a  performance  evaluation  is  the  responsibility  of  the 
sampler.   The  appraiser  uses  the  information  on  the  sampled 
indicators  and  transforms  the  observed  values  into  scores. 
The  goal  of  the  evaluation  process  is  to  provide  a  measure  of 
the  performer  based  on  performance  and  outcome  [Ref .  32] . 
The  manager  must  get  each  of  these  individuals  in  the  decision- 
making process  and  coordinate  their  activities.   Emphasis  is 
placed  in  this  model  on  communications  among  the  right-holders. 
The  right-holders  are  those  given  authorized  power  or  authority 
rights  by  the  organization.   The  authority  rights  are  granted 
to  the  allocator,  criteria  setter,  sampler  and  appraiser. 
There  are  a  variety  of  interconnecting  links  which  may  exist 
between  these  rights-holders  which  define  the  authority  struc- 
ture within  any  given  organization.   Thus  the  formal  evalua- 
tion process  is  based  on  the  structure  of  the  interconnection 
among  the  right-holders  [Ref.  32 J . 

The  Evaluation  and  Authority  Feedback  Model  describes 
performance  by  direct  and  indirect  measures.   Examples  could 
be  the  number  of  units  produced  as  a  measure  of  labor  produc- 
tivity (direct) .   An  indirect  measure  is  the  number  of  professional 
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papers  written  to  evaluate  the  productivity  of  a  scientist  or 
scholar  [Ref.  32].   The  variety  of  samples  requires  judgment 
on  the  part  of  the  evaluator  to  determine  what  is  high  or  low 
performance  so  judgment  must  be  used  by  the  evaluator.   Thus, 
accurate  appraisal  of  the  sample  taken  requires  a  complete 
knowledge  of  the  task,  the  performance,  and  the  specific 
circumstances.   Based  on  all  of  these  measures  the  decision 
is  then  made  on  what,  and  if  anything,  should  be  conveyed  to 
the  performer  concerning  quality  of  the  task  performed  [Ref. 
32]  . 

3 .   Practice  of  Program  Evaluation 

Anderson  and  Ball  take  an  applied  approach  to  program 
evaluation.   They  identify  six  specific  purposes  of  program 
evaluation  [Ref.  33]  : 

1.  to  contribute  to  decision  about  program  installation, 

2.  to  contribute  to  decisions  about  program  continuation, 
expansion,  or  certification, 

3.  to  contribute  to  decisions  about  program  modifications ,* 

4.  to  obtain  evidence  to  rally  support  for  a  program, 

5.  to  obtain  evidence  to  rally  opposition  to  a  program, 

6.  to  contribute  to  the  understanding  of  basic  psychologi- 
cal, social,  and  other  programs.   [Ref.  33:  p.  4] 

Anderson  and  Ball  emphasize  the  practice  aspect  that 
evaluations  can  address  a  wide  variety  of  questions  and  pro- 
vide many  useful  services.   Their  experience  is  primarily  with 
social  programs,  but  it  is  applicable  to  this  study.   They 
identify  a  total  of  seven  types  of  evaluation  methods : 


49 


1.  The  experimental  and  quasi-experimental  study, 

2.  Correlation  methods, 

3.  Surveys, 

4.  Personnel  or  client  assessment, 

5.  Systematic  expert  judgment, 

6.  Clinical  or  case  studies, 

7.  Informal  observation  and  testimony. 

The  real  richness  of  ideas  presented  by  Anderson  and 
Ball  involve  their  emphasis  on  feedback  which  they  refer  to 
as  the:   Communication  and  Dissemination  of  Results.   They 
emphasive  the  need  for  communications  of  the  results  to  be 
bi-directional.   The  effective  communications  of  results  are 
viewed  as  a  sign  of  the  positive  health  of  the  program.   Evalu- 
ators  should  take  pride  in  presenting  and  disseminating  their 
results.   The  dissemination  is  more  than  merely  a  phase  tacked 
on  the  end  of  a  program;  it  involves  more  than  simply  telling 
the  findings.   At  a  minimum,  what  the  dissemination  involves 
is  that  the  results  should  include  information  about  the  evalu- 
ation plans  and  procedures  as  well  as  the  findings.   There  are 
also  several  criteria  provided  to  guide  the  dissemination  to 
various  audiences.   First,  if  an  evaluation  is  worth  doing, 
other  groups  than  the  evaluated  organization  have  some  inter- 
est in  finding  out  about  the  results.   Second,  is  that,  given 
different  audiences,  several  mediums/methods  of  dissemination 
may  be  called  for.   Anderson  and  Ball  classify  some  fifteen 
different  audiences  that  should  at  least  be  considered  for 
dissemination  of  results  [Ref .  33]  . 
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Use  of  results  is  another  consideration  once  the  evalu- 
ation is  completed.   The  logical  expectation  is  that  the 
decision  maker  will  use  the  results  to  make  rational  future 
decisions.   Weiss  lists  several  reasons  why  that  is  not  alv/ays 
the  case.   The  nature  of  the  organization  is  som.etimes  opposed 
to  the  use  of  the  results;  however,  if  the  decision  maker  and 
the  evaluator  have  maintained  close  communications  throughout 
the  process,  the  potential  objections  can  be  reduced  [Refs.  33, 
34] .   Particular  consideration  must  be  given  not  only  to  factual 
evaluation  of  each  program,  but  also  the  salesmanship  aspect 
of  presenting  the  evaluations. 

Anderson  and  Ball  have  adapted,  from  a  paper  by  J.  S. 
Berke  a  structure  which  they  suggest  should  be  applied  when 
writing  evaluation  reports. 

•Brevity  and  clarity.   Critical  findings  should  be 
summarized  clearly  and  simply  at  the  outset. 

•Timeliness — to  be  useful  and  utilized,  results  must 
be  reported  according  to  other  peoples'  schedules  and 
not  the  evaluator 's  research  clock. 

•Interim  products  and  reports--these  help  prepare 
decision  makers  for  the  impact  of  larger,  later  evalua- 
tion reports.   Besides,  they  can  allow  preliminary 
planning  for  utilization  even  before  the  final  report 
is  available. 

•Responsiveness — he  notes  here  that  traditional  researchers 
tend  to  make  questions  more  interesting,  design  more 
elegant,  analysis  more  comprehensive,  and  utilization 
of  recommendations  more  guarded  than  necessary. 
[Ref.  33:  pp.  107-108] 

In  summary  the  program  evaluator  has  the  responsibility 

to  push  the  results  of  the  evaluation  and  to  bring  them  to 

the  attention  of  all  those  who  should  be  made  aware  [Ref.  33] . 
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4 .   Critical  Incident  Evaluation 

Critical  Incident  Evaluation  Program  involves  manage- 
ment writing  regular  reports  on  the  performance  they  observe. 
The  critical  incident  process  has  as  its  roots  the  idea  that 
actual  behavior  should  be  appraised,  not  traits  of  behavior. 
It  is  an  attempt  to  justify  ratings  based  on  specific  incidents 
that  provide  support  evidence  for  the  evaluation.   Levinson 
places  strong  emphasis  on  the  need  to  evaluate  on  how  things 
get  done  and  not  just  results  [Ref.  36].   Levinson  argues  that 
in  reality  people  are  evaluated  on  the  "how",  but  many  are 
led  to  believe  that  they  are  just  judged  based  on  results. 

Thus,  the  profit  oriented  manager  pulls  his  company 
out  of  the  "red",  but  is  criticized  for  the  methods  used  to 
do  it.   Levinson  cites  many  deficiencies  with  current  evalua- 

« 

tion  systems.   One  is  that  no  matter  how  well-defined  the  dimen- 
sions for  appraising  performance  on  a  quantitative  basis, 
judgments  on  performance  are  always  subjective  and  impression- 
istic [Ref.  36].   Second,  delay  in  giving  feedback  creates 
frustration  both  when  performance  is  good  and  credit  is  de- 
served, and  when  performance  is  bad  and  criticism  is  rendered 
long  after  the  performance.   Levinson  views  performance  evalua- 
tion not  as  a  technique,  but  a  process  involving  both  people 
and  data.   Thus,  the  process  is  inadequate  [Ref.  36 J . 

Levinson  proposes  that  an  effective  critical  incident 
model  be  composed  of  the  following  [Ref.  36]: 

1.   A  dynamic  job  description — one  which  amplifies 
statements  of  job  responsibility  and  desired 
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outcome  by  describing  the  emotional  and  behavioral 
topography  of  the  task. 

2.  A  critical  incident  process — this  requires  jointly 
setting  objectives  and  discussing  each  piece  of 
behavior  that  is  judged  good  or  bad  by  the  manager. 

3.  A  psychological  support  system — to  accomplish 
this,  the  manager  must  learn  to  cope  with  feelings 
or  guilt  over  the  appraisal.   He  calls  this  upward 
appraisal  concept  in  which  managers  who  develop 
employees  through  their  effective  appraisal  should 
be  compensated   [Ref.  36] 

An  interesting  aspect  was  observed  by  Winton-Oberg 

while  observing  the  use  of  the  critical  incident  process  at 

General  Electric: 

People  who  received  honest  but  negative  feedback  are 
typically  not  motivated  to  do  better  and  often  do 
worse  after  appraisal  interviews.   [Ref.  35:  p.  64] 

The  whole  critical  incident  process  involves  a  con- 
tinuous flow  of  feedback  to  the  evaluated  employee. 

Although  the  critical  incident  process  by  its  very 
nature  should  lead  to  a  continuous  review  and  communication 
on  positive  and  negative  behavior,  that  is  not  always  the 
case.   Also  the  critical  incident  itself  is  often  a  subjec- 
tive evaluation  on  the  part  of  the  supervisor.   The  system 
should  focus  attention  on  actual  behavior  rather  than  on 
employee  traits.   Also  it  provides  an  opportunity  for  the 
employee  to  find  out  specifically  how  to  perform  if  he  wants 
to  be  rated  higher  the  next  time.   This  system  does  encourage 
regular  feedback  since  the  supervisor  can  hardly  wait  to  the 
end  of  a  period  to  feedback  what  he  sees  as  a  critical  inci- 
dent [Ref.  35].   One  of  the  major  factors  supporting  this 
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system  is  that  the  critical  incident  process  takes  the  sur- 
prise out  of  the  annual  or  semi-annual  evaluation.   All  those  . 
incidents  viewed  by  the  supervisor  as  critical  would  have 
already  been  brought  to  the  attention  of  and  discussed  with 
the  subordinate.   This  technique  provides  more  regular  feed- 
back to  the  subordinate  because  both  good  and  bad  incidents 
should  be  noted  continually.   Also  if  the  employee  feels  un- 
fairly judged,  he  may  appeal  the  criticism  immediately  rather 
than  waiting  a  long  period  of  time  [Ref.  36]. 

E.   SPECIFIC  EVALUATION  MODELS  APPLIED  TO  THE  MILITARY 

The  approaches  to  the  evaluation  process  discussed  thus 
far  have  been  general  in  nature.   However,  in  this  study  evalu- 
ation processes  specific  to  military  environments  were  also 
reviewed.   Following  are  four  evaluation  approaches  to  the 
military  which  were  reviewed  in  detail. 

1.   Army  Training  and  Evaluation  Program  (ARTEP) 

The  Army  Training  and  Evaluation  Program  (ARTEP)  for 
Infantry  Battalions  was  reviewed  during  this  research.   In 
addition  a  detailed  brief  was  provided  by  the  G-3 ,    7th  Infantry 
Division  [Ref.  37].   The  Army  Program  specifics  are  laid  out 
for  Infantry,  Airborne,  Air  Assault,  and  Ranger  units  in  one 
volume  [Ref.  3  8] .   That  guide  is  further  supported  by  local 
guidance  at  each  command. 

The  Infantry  ARTEP  Evaluation  has  four  purposes:   (other 
references  exist  for  ARTEP  application  to  Armor,  Mechanized 
units,  etc. ) : 
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1.  Establish  infantry  unit  training  missions  with 
specified  tasks,  conditions,  and  standards  of 
performance  for  combat-critical  missions. 

2.  Under  simulated  combat  conditions,  train  and 
evaluate  the  ability  of  the  unit. 

3.  Evaluate  the  effectiveness  of  past  training  of  all 
echelons  of  the  Battalion. 

4.  Assess  future  training  needs.   [Ref.  38] 
The  evaluation  is  broken  into  three  phases: 

1.  Critique  of  Leaders 

2.  Evaluation  of  Whole  Battalion 

3.  Written  Report 

The  ARTEP  employs  the  following  philosophy: 

1.  The  evaluation  is  conducted  two  levels  down  (Division 
evaluates  Battalion) 

2.  Battalion  ARTEP • s  are  scheduled  every  18  months. 

3.  Companies/Platoons/Squadrons  are  evaluated  annually. 

4.  Subunit  evaluators  focus  on  the  end  evaluation. 

5.  Evaluations  are  scheduled  in  sequential  progression 
(squad,  platoon,  company,  then  battalion) 

6.  Scheduled  preparation  time  is  five  weeks  for  a  Battalion 

7.  Fence  off  the  unit  for  two  weeks  following  the  exer- 
cise (no  other  requirements  are  placed  on  the  unit 
during  this  period) 

8.  Use  operating  force  tactics 

9.  Do  not  evaluate  during  other  training  or  field 
exercises.   [Ref.  37] 
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The  ARTEP  Evaluations  are  strictly  conducted  within  the 
divisional  unit.   The  Assistant  Division  Commander  is  desig- 
nated "exercise  director,"  but  a  sister  battalion  is  desig- 
nated as  the  force  to  provide  the  senior  evaluator,  another 
lieutenant  colonel  as  well  as  company  and  other  evaluators . 
The  ARTEP  is  viewed  as  a  training  evaluation. 

The  ARTEP  System  provides  a  combination  of  both  objec- 
tive and  subjective  feedback  to  the  evaluated  unit.   All  of 
the  tasks  evaluated  in  the  ARTEP  evaluation  are  reported  on  a 
Satisfactory/Unsatisfactory  basis.   All  unsatisfactory  or  not 
evaluated  ratings  must  be  explained  in  the  remarks.   This 
data  is  placed  on  a  standardized  data  collection  sheet.   A 
copy  is  provided  to  the  evaluated  unit  shortly  after  the 
exercise  is  completed.   Additionally  a  copy  of  the  data  sheet 
is  forwarded  (without  any  identifying  unit  data)  to  the  U.S. 
Army  Infantry  School,  Fort  Benning,  GA.   The  data  forwarded 
to  higher  headquarters  is  for  the  purpose  of  future  training 
development  efforts  and  for  use  in  improving  the  ARTEP  doc- 
trine, devices,  and  techniques  Army-wide  [Ref .  3  8] .   The  over- 
all evaluation  given  to  a  unit  is  a  Satisfactory/Unsatisfactory 
No  percentile  scores  are  associated  with  or  computed  for  the 
ARTEP  results.   Thus  no  comparison  is  made  from  unit  to  unit 
using  ARTEP  data  {Ref.  37] . 

In  addition  to  the  written  report,  a  continuous  review 
and  exchange  of  comments  between  the  evaluator  and  unit  is 
encouraged.   Shortly  after  the  ARTEP  is  completed,  an  oral 
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debrief  is  held  for  all  key  personnel  of  the  battalion  and 
the  evaluators .   In  some  cases  the  detailed  (good)  evaluations 
of  units  to  the  squad  level  are  announced  to  all  of  the  troops 
at  formation.   This  technique  is  viewed  as  a  positive  motivation 
for  the  small  units  and  their  leaders  [Ref .  37] . 
2 .   Funk  Conceptual  Model  of  Unit  Performance 

A  recent  study  by  Steven  Funk  [Ref.  11]  attempted  to 
model  combat  unit  effectiveness  for  the  U.S.  Army.   He  found 
many  of  the  same  difficulties  in  defining  effectiveness  that 
this  author  found  in  defining  readiness.   He  described  effec- 
tiveness as  follows: 

Individual  evaluators  determine  unit  effectiveness 
based  upon  some  explicit  or  intuitive  constraint  of  what 
units  are  supposed  to  do  and  how  that  is  achieved. 
Unit  effectiveness  is  determined  by  evaluating  both 
outcomes  and  processes,  and  is  determined  for  perfor- 
mance on  all  tasks  presented  to  the  unit  whether  they 
are  combat  related  or  not.   [Ref.  11:  p.  19] 

He  interviewed  groups  of  officers  and  senior  enlisted, 

primarily  battalion  commanders,  company  cominanders ,  staff 

officers,  unit  officers,  and  first  sergeants.   He  also  reviewed 

a  wide  array  of  existing  predictive  models  from  the  Army,  Navy, 

and  Air  Force.   He  identified  early  readiness  studies  such 

as  Army  Training  Study  (ARTS)  19  77-19  78,  which  concluded  that 

readiness  should  be  viewed  from  a  perspective  of  relationships 

among  personnel,  weapons,  equipment,  resources,  detractors, 

and  incentives.   He  also  looked  at  the  Navy's  19  74  report  on 

readiness  which  stated,  "Organizational  effectiveness  implies 
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an  evaluative  or  judgmental  process  against  an  expectation  or 
standard"  [Ref.  11:  p.  11]. 

The  result  of  his  studies  concluded  that  military  unit 
effectiveness  is  far  more  readily  conceptualized  than  measured 
[Ref.  11] .   Part  of  the  problem  of  measuring  military  unit 
effectiveness  is  operationally  defining  readiness  in  a  peace- 
time environment. 

Fun>c  referenced  Etzioni  who  noted  that  organizations 
attempt  to  achieve  a  balanced  distribution  of  resources  across 
needs  and  did  not  attempt  to  maximize  satisfaction  in  one 
area  [Ref.  11] .   Funk  concluded  that  military  units  in  peace- 
time have  great  difficulty  in  maximizing  effectiveness  or 
readiness  because  they  really  face  not  one  objective,  but  a 
whole  array  of  competing  priorities  all  calling  for  their 
resources.   His  studies  resulted  in  the  proposed  conceptual 
model  in  Figure  3.4. 

This  model  is  referred  to  as  the  Unit  Performance  Sys- 
tems model.   It  describes  behavior  as  the  interrelationships 
of  technology,  formal  and  informal  unit  structure,  perceived 
unit  requirements  and  priorities,  available  resources,  unit 
climate,  unit  process,  cause  of  evaluation  of  unit  actions, 
and  the  unit  operating  environment  [Ref.  11] . 

3.   Hayes  et  al.  Statistical  Model  of  Combat  Effectiveness 

In  19  77,  a  cross-sectional  study  for  the  Defense  Ad- 
vanced Research  Projects  Agency  was  conducted.   An  effort  was 
made  to  identify  through  detailed  statistical  methods  key 
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Figure  3.4.   Unit  Performance  Systems  Model 
[Ref.  11:  p.  38] 
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factors  which  contributed  to  levels  of  success  in  22  combat 
engagements  ranging  from  World  War  II  to  Vietnam.   The  study- 
was  aimed  at  determining  the  effectiveness  of  infantry  battalions 
based  on  the  judgment  of  experienced  Marine  Corps  officers  in- 
volved in  these  engagements.   Multi-variate  statistical  analy- 
sis was  used  to  examine  the  data  results.   The  study  revealed 
that  adaptive  behavior  by  units,  that  is  reaction  to  the  com- 
bat environment,  was  the  single  most  important  discriminator 
between  successful  and  unsuccessful  performance.   They  also 
identified  three  types  of  activity  which  appeared  to  be  closely 
related  to  mission  accomplishment:   command  and  planning 
(strongly  related) ,  supporting  fires  (moderately  associated) , 
and  coordination  function  (relatively  weak  and  appeared  to  re- 
quire effective  command  and  planning  before  it  made  a  differ- 
ence) [Ref .  17] . 

This  study  defined  "combat  effectiveness"  as 

. . .the  ability  of  a  unit  to  accomplish  a  military 
mission.  As  such,  combat  effectiveness  refers  to 
performance  in  a  hostile  environment.   [Ref.  17:  pp.  1-2] 

As  such  the  focus  on  the  effectiveness  of  a  military  unit  is 
placed  on  outcomes  and  effectiveness  and  can  only  be  measured 
by  mission  accomplishment.   The  infantry  battalion  was  seen 
as  not  operating  in  a  vacuum,  but  rather  in  a  world  of  obsta- 
cles and  constraints.   Thus  the  adaptability  was  an  important 
element  in  performance.   Besides  adaptability,  Hayes  identified 
a  number  of  key  elements  which  included: 

•maneuver  during  the  action 

•preparatory  air,  artillery,  and  naval  gunfire 
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•  communications 

•quality  of  planning  and  information  as  well  as  others, 
[Ref.  17:  pp.  1-18] 

This  study  was  done  during  the  period  when  MCCRES  was 
being  developed.   One  important  observation  made  here  with 
regard  to  feedback  was  that,  although  MCCRES  is  comprehensive 
and  standardized,  a  great  deal  of  information  is  currently 
passing  through  the  hands  of  the  evaluators  that  could  be  ex- 
tremely valuable  to  the  evaluated  units  [Ref.  17] . 

4 .   Sassone  Economic  Model  of  Training  Effectiveness 

A  fourth  approach  to  effectiveness  is  taken  by  Sassone, 
who  has  taken  an  economic  viev;  which  is  different  from  the 
traditional   economic  approach  to  evaluating  military  training 
programs.   The  traditional  approach  is  to  evaluate  the  train- 
ing on  its  projected  costs  and  benefits.   The  Sassone  approach 
requires  that  any  new  training  program  be  compared  directly 
with  existing  programs  that  it  will  replace  or  amplify.   Tra- 
ditionally equipment  and  training  are  judged  simply  on  their 
ability  to  function  for  the  intended  purpose.   The  training 
programs  are  particularly  difficult  to  evaluate  because  of 
necessity  of  measuring  the  impact  of  the  training  on  the 
trainee.   In  the  private  sector  training  programs  are  gener- 
ally judged  on  their  effectiveness  by  the  change  in  lifetime 
earnings  of  the  trainee  after  the  program  [Ref.  39].   If  the 
cost  of  the  training  program  is  less  than  the  lifetime  increase 
in  wages,  then  this  program  is  judged  to  be  economically 
worthwhile.   However,  training  programs  in  the  military  cannot 
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be  judged  this  same  way,  because  there  is  no  way  to  measure 
their  contribution  toward  readiness,  sustainability ,  or  effec- 
tiveness since  these  are  not  commodities  or  traded  in  the 
market  place  [Ref.  39]. 

Sassone's  methodology  requires  that  when  a  new  type  of 
training  for  an  individual  or  unit  is  developed,  it  must  be 
stated  in  specific  terms  to  what  extent  it  substitutes  for 
existing  programs.   The  constraint  in  this  case  is  the  train- 
ing budget  for  the  specific  type  of  training.   There  are  three 
steps  in  this  methodology  [Ref.  39] . 

(1)   An  equation  relating  training  inputs  and  outputs  must 
be  developed.   This  requires  a  comparison  of  the  two  types 
of  training  under  consideration.   (See  Figure  3.5.) 


Ef  f ectivei  less 


Effective]  less 
tvel  70^ 


Effectiveiiess 


J^   Training  type  2       Training  type  1 


Source:      Adapted   from    [Ref.    39:    p.    40] 


Figure  3.5. 


A  Hypothetical  Training  Input  vs .  Output 
Relationship 


62 


The  figure  shows  different  levels  of  effectiveness  which  can 
be  achieved  with  various  combinations  of  the  two  types  of 
training.   The  curve  shows  all  the  levels  of  effectiveness 
which  can  be  achieved  with  the  various  combinations.   The 
shape  of  the  curves  describe  the  substitute  ability  of  one 
training  type  for  the  other. 

(2)  Relationship  of  cost  and  use  data  must  be  obtained. 
There  are  three  types  of  costs  involved:   front  end  costs  for 
the  training  program  under  evaluation;  operational  costs,  and 
training  budgets  for  the  training  manager  of  the  program. 

(3)  The  last  step  requires  the  development  of  a  relation- 
ship between  training  and  effectiveness. 

Where  potentially  greater  effectiveness  is  available 
at  greater  cost,  the  issue  is  simply  whether  the 
greater  effectiveness  is  worth  the  greater  cost. 
[Ref.  39:  p.  41] 

Sassone  fails  to  specify  how ' this  relation  of  training  and 
effectiveness  is  obtained.   It  is  assumed  by  this  researcher 
that  it  refers  to  some  evaluation  method  such  as  MCCRES . 
Measures  such  as  those  generalized  by  ARTEP  allow  for  no  rela- 
tive measure  of  effectiveness  which  is  required  for  this 
type  of  comparison. 

The  cost  effectiveness  equation  is  then  developed  by 
optimum  use  of  available  training  resources  at  each  level  of 
the  training  budget.   The  term  "optimum"  in  Sassone 's  proposal 
is  one  that  seems  to  be  in  conflict  with  other  theorists. 
Funk  [Ref.  11]  for  instance  indicates  that  many  competing 
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priorities  take  resources  from  the  unit  thus  precluding 
optimality  of  resource  application. 

Figure  3.6  represents  a  relation  between  cost  and  a 
sample  program. 


Initial  training 

vs.  effectiveness 


:  5udget 
military  value 


^  Level  of  effectiveness 

Source;   Adapted  from  [Ref.  39:  p.  41] 

Figure  3.6.   The  Military  Value  of  a  Training  Program 
Figure  3.6  is  described  by  Sassone  as  starting  at  a 
minimal  level,  indicating  that  some  level  of  effectiveness 
would  exist  even  without  training.   The  curve  initially  rises 
rapidly  but  starts  to  level  off  as  maximum  effectiveness  is 
reached.   As  better  training  programs  are  conceived,  the  mid- 
dle of  this  curve  would  shift  up.   If  some  new  program  were  a 
perfect  substitute  of  an  existing  one,  the  new  curve  would 
overlay  the  existing  one  and  be  exactly  the  same  curve.   Various 
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shifts  in  the  curve  up  can  be  caused  by  a  greater  level  of 

effectiveness  achieved  for  a  lower  cost.   The  Figure  3.6  shows 

how  the  military  value  of  various  training  programs  can  be 

compared  with  other  or  existing  programs.   In  addition  one 

must  consider  opportunity  costs,  funds  used  for  research  and 

development  of  new  programs  which  the  service  could  otherwise 

have  devoted  to  existing  programs  [Ref.  39], 

The  military  value  of  the  new  training  program  is  repre- 
sented by  the  increment  in  the  current  training  budget 
needed  to  increase  effectiveness  by  the  same  amount 
as  the  increase  in  effectiveness  associated  with  the 
new  program.   [Ref.  39:  p.  42] 

The  opportunity  cost  of  a  new  military  training  program  is 
determined  by  spreading  the  front  end  costs  equally  over  the 
units  trained  per  year  and  the  expected  life  time  of  the 
program  applying  the  appropriate  discount  rate.   Figure  3.6 
is  used  to  compare  the  'military  value  of  a  training  program  by 
using  the  cost-effectiveness  ratio:   if  the  military  value 
is  less  than  the  opportunity  cost,  then  the  new  program  would 
not  be  considered  economically  feasible  [Ref.  39] . 
5.   Summary 

This  section  reviewed  four  approaches  to  evaluation 
that  have  recently  been  developed  for  applications  to  the  mili- 
tary.  The  ARTEP  employed  [Ref.  38]  by  the  Army  provided  a 
process  of  evaluation  which  has  much  in  common  with  MCCRES 
especially  in  its  application  to  similar  organizations.   The 
conceptual  model  by  Funic  [Ref.  11]  provided  a  view  not  only 
of  the  elements  involved  in  training,  but  also  of  some  of  the 
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barriers  and  constraints  that  units  must  face  in  a  peacetime 
environment.   The  results  of  the  Hayes  et  al..  Statistical 
Model  [Ref.  17]  were  valuable  in  that  specific  characteristics 
were  identified  that  led  to  success  in  actual  combat  engage- 
ments.  Finally  the  Sassone,  Economic  Training  Model  was  pre- 
sented [Ref.  39]  which  proposed  some  new  thoughts  on  the  sub- 
ject of  evaluating  military  training  programs.   The  ideas  brought 
forth  in  these  studies  will  be  of  benefit  in  the  future  examina- 
tion of  MCCRES . 

F.   FEEDBACK 

...studies  have  shown  that  accurate  feedback  about 
quality  of  worlc  is  a  strong  stimulus  to  good  perfor- 
mance..  .People  worlc  better  when  they  know  how  well 
they  are  doing  in  relation  to  some  meaningful  standard. 
[Ref.  43:  p.  174] 

This  final  section  of  the  chapter  will  discuss  feedback. 
Feedback  is  variously  described  by  different  writers  as  feed- 
back [Refs.  25,27,36,41],  feedback  control  [Ref.  15],  communi- 
cations [Ref.  32]  and  further  as  dissemination,  communications, 
and  utilization  [Ref.  33] . 

Whatever  the  feedback  is  called  it  is  common  to  all  of 

the  models  discussed.   Feedback  is  a  necessary  part  of  all 

evaluation  systems. 

Control  to  be  effective,  requires  timely,  accurate, 
and  dependable  indicators  of  effectiveness.   Feedback 
should  provide  information  that  is  adequate  to  suggest 
appropriate  action.   To  that  end,  reports  must  point 
to  significant  developments,  as  distinguished  from 
what  is  normal,  usual,  and  to  be  expected.   [Ref.  30: 
p.  504] 
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Flamholtz  [Ref.  43]  has  presented  a  typical  model.  Figure 
3.7  of  an  organizational  control  system  which  places  emphasis 
on  the  feedback  loop.   He  describes  feedback  as  a  necessary 
managerial  function  of  the  control  systems  which  identifies 
problems  [Ref.  43] . 
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Source:   Adopted  from  [Ref.  43:  p.  56] 

Figure  3.7.   Model  of  Organizational  Control  System 

This  model  provides  a  typical  example  of  how  the  essential 
feedback  loop  functions  in  an  organizational  control  or  eval- 
uation system.   The  feedback  loop  provides  an  interactive 
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process  that  allows  for  routine  recycling  of  the  results  as 
a  comparison  of  the  observed  performance  to  the  measurement 
system.   The  essential  element  to  observe  here  is  that  feed- 
back is  a  standard  fixture  of  the  evaluation  system.   All 
systems  viewed  that  provide  a  diagrammatic  model  include  a 
feedback  mechanism.   In  addition  Anderson  and  Ball  and  the 
ARTEP  provided  detailed  structures  for  feedback. 

G.   SUMMARY 

This  chapter  has  covered  the  topics  of  management,  manage- 
ment control,  evaluation,  and  feedback.   These  topics  can  be 
seen  as  moving  from  the  general  to  the  specific.   The  bound- 
aries between  these  topics  are  not  clear  cut.   There  is  a 
great  deal  of  overlap  in  the  fields  of  management,  organiza- 
tional control,  evaluation,  and  feedback.   This  is  particularly 
true  when  they  are  viewed  from  different  disciplines.   For 
instance,  much  of  what  is  classified  in  education  as  evaluation, 
would  be  classified  under  control  in  organizational  or  mana- 
gerial theory.   A  similar  situation  exists  with  regard  to 
terminology  for  feedback.   Theorists  in  both  the  same  and 
different  disciplines  often  use  various  terms  to  describe 
feedback. 

Since  MCCRES  attempts  to  measure  or  quantify  readiness, 
the  military  models  were  included.   Although  the  last  three 
are  static  measures  or  prediction  of  effectiveness  as  opposed 
to  MCCRES  and  ARTEP  which  are  dynamic  measures,  they  do 
provide  insight  into  the  various  views  of  effectiveness  and 
the  observations  made  from  several  different  disciplines. 
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Feedback  was  discussed  with  emphasis  on  relating  the  many 
different  terms  used  to  describe  feedback.   In  addition  the 
feedback  loop  which  is  common  to  all  models  reviewed  has  been 
discussed.   The  theory  of  feedback  is  essential  to  this  study 
and  the  many  ideas  and  structures  placed  on  it  will  give  much 
insight  when  applied  to  MCCRES.   Certain  elements  such  as: 
brevity  and  clarity,  timeliness,  interim  products  and  re- 
ports, and  responsiveness  [Ref.  33]  have  almost  universal 
application  to  feedback  theory.   The  MBO ,  Evaluation  and 
Authority,  Critical  Incident,  and  ARTEP  all  provide  a  wealth 
of  insight  on  how  feedback  techniques  should  be  applied. 

The  next  chapter  will  provide  a  detailed  overview  of  the 
MCCRES  System. 
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IV.   MCCRES 

In  the  last  chapter  detailed  examination  was  made  of 
management  control,  four  general  evaluation  systems,  and  four 
specific  evaluation  systems  which  apply  to  the  military.   This 
should  provide  the  reader  with  a  background  from  which  to  view 
MCCRES.   In  this  chapter  attention  is  given  in  detail  to  the 
MCCRES  itself,  its  development,  explanation  of  the  system's 
mission  performance  standards,  and  the  evaluation  process 
applied  by  MCCRES  as  well  as  the  reports  and  feedback  designed 
into  the  current  system.   The  goal  of  this  chapter  is  to  pro- 
vide the  reader  with  a  general  understanding  of  how  the  MCCRES 
is  applied,  based  on  the  most  current  Marine  Corps  directives. 

The  purpose  of  the  Marine  Corps  Combat  Readiness  Evalua- 
tion System  (MCCRES)  is  to  provide  a  timely  and  accurate 
evaluation  of  readiness  of  Fleet  Marine  Forces,  includ- 
ing reserve  units,  to  accomplish  assigned  missions. 
[Ref.  1:  p.  1-A-l] 

The  formal  MCCRES  must  be  given  at  least  once  every  two  years 

for  all  FMF  units   for  which  performance  standards  have  been 

written. 

A.   DEVELOPMENT  AND  STRUCTURE 

The  development  of  the  MCCRES  was  begun  with  a  review  of 
the  various  combat  readiness  measurements  then  currently  in 
use.   Many  of  these  consisted  of  individual,  as  well  as  unit, 
evaluations  and  were  in  the  form  of  inspections  which  stressed 
appearance  more  than  combat  proficiency  [Ref.  1]. 
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The  MCCRES  was  structured  to  access  the  ability  of  all 
Marine  Corps  Combat  units:   air  and  ground  combat,  combat 
support,  and  combat  service  support.   Specifically  it  was 
designed  to  provide: 

-Performance  standards  based  on  assigned  missions 

-A  standardized  evaluation  process 

-A  standardized  reporting  system 

-Feedback  to  units  indicating  strengths  and  weaknesses. 
[Ref.  1:  p.  l-A-2] 

The  MCCRES  was  designed  to  provide  a  detailed  analysis  of 
the  units'  operational  capabilities  and  to  specifically  iden- 
tify strengths  and  weaknesses  through  comparison  with  doctrine 
under  simulated  combat  conditions.   The  doctrine  is  embodied 
in  a  checklist  of  requirements.   The  hierarchy  of  these  quali- 
ties is  identified  from  general  to  specific  as  Mission  Per- 
formance Standards  (MPS) ,  each  of  which  consists  of  three 
parts:   Tasks,  Conditions,  and  Requirements.   The  MPS  set  the 
Commandant's  acceptable  standards  for  tactical  performance 
throughout  the  Marine  Corps.   Based  on  comparison  of  the 
unit's  performance  to  the  MPS  under  simulated  conditions,  a 
determination  is  made  of  "Combat  Ready/Not  Combat  Ready." 
The  MCCRES  is  designed  for  both  formal  and  informal  evaluation. 
The  informal  evaluation  can  be  provided  by  the  unit  itself 
measuring  its  performance  standards.   The  formal  evaluation  is 
accomplished  by  higher-level  commands  normally  evaluating  a 
selection  of  MPS  .   For  the  formal  evaluation  certain  MPS 
are  required  in  all  evaluations  and  others  are  selected  by  the 
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organization  directing  the  evaluation  based  on  detailed 
guidance  in  Ref.  1.   A  cross  section  of  MPS's  are  used  to 
provide  a  statement  of  the  organization's  operational  readiness 

The  evaluators  employed  in  the  formal  system  must  be  ex- 
perienced, capable  personnel.   The  overall  system  is  designed 
to  identify  specific  operational  deficiencies  and  the  nature 
of  any  latent  or  potential  problems.   The  MCCRES  cycle  is  des- 
cribed in  Figure  4-1. 
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Figure  4.1.   MCCRES  Cycle 


72 


B.   MISSION  PERFORMANCE  STANDARDS  (MPS) 

In  building  the  MCCRES  framework  it  was  decided  that  the 
evaluation  would  be  structured  around  a  set  of  Mission  Per- 
formance Standards  that  establish,  on  a  Marine  Corps-wide 
basis,  acceptable  operational  performance.   Derivation  of  these 
standards  was  accomplished  by  using  five  interrelated  concepts 
concerning  determination  of  combat  readiness.   The  concepts 
are: 

-  Standards  must  be  objective 

-  Standards  must  define  for  the  evaluator  what  quality  means 

-  Standards  must  be  based  on  published  doctrine 

-  Standards  must  involve  the  performance  of  individual 
Marines  in  evaluation  of  unit  combat  readiness 

-  Standards  must  be  simple  for  the  evaluator.   [Ref .  IJ 
The  MPS  are  defined  in  detail  in  Volumes  II  through  VIII 

of  Marine  Corps  Order  3501.2.   Each  of  these  volumes  pertains 
to  a  specific  type  of  unit  to  be  evaluated,  for  instance 
Volume  II  pertains  to  Infantry  and  Volume  III  to  Rotary  Wing 
Observation  Squadrons.   In  this  way  the  volume  lists  the  MPS's 
involved  for  each  unique  type  of  unit. 

Each  MPS  is  further  broken  down  into  three  major  parts: 

-"The  Task  to  be  performed 

-The  Condition  under  which  the  task  is  to  be  performed 

-The  Requirements  which  must  be  accomplished  to  successfully 
fulfill  the  task."  [Ref.  1:  p.  I-B4] 

Examples  of  the  MPS,  Tasks,  Conditions,  and  Requirements 

are  provided  in  Figure  4.2. 
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To  assist  the  Evaluator  when  the  Requirement  does  not 
completely  define  the  quality  of  performance  necessary,  Key 
Indicators  (KI)  are  provided.   These  KI's  are  defined  to  clarify 
and  explain  the  requirements  [Ref.  1].   The  Key  Indicator  (KI) 
is  further  defined  in  Figure  4.3,  Key  Indicators  for  Evaluator. 

For  simplicity  and  to  reduce  the  need  for  subjective  judg- 
ments, all  MPS's  were  designed  to  permit  the  Evaluator  only 
three  possible  outcomes  for  each  requirement:   Yes/No /Not 
Applicable.   Those  MPS's  dealing  with  the  performance  of  indi- 
vidual Marines,  Exercise  of  Command  and  Control,  and  Fire 
Support  Coordination  must  be  used  during  every  evaluation.   The 
reason  that  all  MPS's  are  not  employed  during  each  evaluation 
is  a  limitation  of  time  and  resources  [Ref.  1]. 

In  addition  to  the  evaluation  of  MPS's  which  use  a  simu- 
lated combat  environment  against  an  aggressor  force,  a  series 
of  MCCRES  Standard  Performance  Tests  (SPTi  have  also  been 
developed.   Examples  of  these  types  of  proficiency  tests  are: 
Foot  Mobility  Test,  Dragon  Gunner's  Test,  Engineer  Route  Recon- 
naissance Test,  and  Aircraft  Recognition  Test.   The  SPT's  are 
used  to  support  the  MPS's  and  are  tested  objectively  prior 
to  the  MCCRES  operational/readiness  test.   The  SPT  is  formated 
exactly  as  the  MPS;  however,  a  'Yes'  must  be  achieved  in  each 
area  before  the  MCCRES  is  continued.   Any  unit  not  found 
proficient  in  the  SPT  portion  of  MCCRES  will  not  be  tested  on 
the  tactical  portion. 
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KEY  INDICATORS  FOR  EVALUATOR 

WARNING  ORDER 

MUST  INCLUDE: 

General  information  on  the  situation. 

Units  to  make  the  move  and  anticipated  sequence. 

Anticipated  time  of  move. 

Anticipated  route  and  destination. 

STAFF  COORDINATION 

MUST  SPECIFICALLY  ADDRESS: 

Route  reconnaissance — map,  physical,  or  aerial  photo. 
Support  needed  from  higher  hq — air,  vehicle,  or  firepower. 
Fire  support  coverage  throughout  movement. 
Cover  and  concealment  available  enroute. 
Logistic  aspects  affected  by  location  change. 

BASIC  PLAN 

MUST  BE  BASED  ON: 

Movement  at  speed  desired  by  higher  hq. 
Control  of  all  elements  during  move. 
Security  of  all  movement  serials. 
Known  enemy  capabilities. 

MOVEMENT  ORDER 
MUST  CONTAIN: 

Clearly  stated  mission. 

Definition  of  all  control  measures  to  be  used:   check  points, 
phase  lines,  march  objectives,  etc. 

Specific  missions  for  attached  and/or  support  elements: 
tanks,  engineers,  TOW,  etc. 

Identification  of  initial  point  from  which  move  will  begin. 

Assignment  to  march  serials. 
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Time  of  departure. 

Clear  identification  of  available  outside  support. 

Detailed  security  procedures. 

As  much  information  on  threat  as  is  available: 

Emphasis  on  specialized  weaponry  that  can  affect  the 
move;  ATGM,  artillery,  air,  etc. 


Source:   Adopted  from  [Ref.  l.P:  p.  I-B-6J 


Figure  4.3.   Sample  of  Key  Indicators  for  Evaluator 
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C.   PERSONNEL  ASSIGNMENT  FOR  THE  EVALUATION 

The  key  to  successful  application  of  each  MCCRES  is  the 
evaluation  process.   Selection  and  training  of  evaluators  is 
at  the  very  heart  of  the  evaluation  process.   Credibility  of 
the  evaluators  is  essential  to  the  success  of  MCCRES.   The 
evaluators  selected  are  to  have  the  requisite  skills  and  recent 
experience  as  to  preclude  the  need  for  a  long,  detailed  school 
for  the  evaluators.   However,  as  needed  and  as  time  allows, 
training  is  provided  to  evaluators.   Since  all  evluation  sys- 
tems require  some  judgment  on  the  part  of  the  evaluators,  and 
MCCRES  is  no  exception,  sound  judgment  is  a  prerequisite  for 
each  evaluator  chosen. 

The  evaluation  is  structured  with  the  Commanding  General's 
Fleet  Marine  Force  Pacific  (FMFPAC)  and  Fleet  Marine  Force 
Atlantic  (FMFLANT)  as  the  Evaluation/Exercise  Commander (s) 
(EC)  for  the  initiation  and  conduct  of  all  formal  MCCRES 
Evaluations .   The  responsibilities  of  the  Evaluation/Exercise 
Commanders  are  outlined  in  detail  in  Ref.  1. 

1.  Evaluation/Exercise  Director  (ED) 

ED  will  be  designated  by  the  Exercise  Commander  (EC) 
and  is  responsible  to  prepare  for,  conduct,  and  report  the  formal 
MCCRES  evaluations.   Normally  the  Coramanging  General  of  the 
respective  Marine  Division  or  Wing  whose  subunit  is  to  be 
evaluated  is  designated  ED. 

2 .  Tactical  Exercise  Controller  (TEC) 

TEC  will  be  designated  by  the  ED,  along  with  an  ap- 
pointed staff,  to  serve  as  the  control  agency  for  conducting 
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the  exercise.   The  TEC  compiles  and  analyzes  the  results  of 
the  evaluation.   The  TEC  is  also  responsible  for  the  detailed 
training  of  the  evaluators  as  well  as  development  of  a  detailed 
exercise  scenario.   At  the  end  of  the  evaluation  a  formal  re- 
port is  prepared  for  the  ED  describing  the  combat  readiness 
of  the  evaluated  unit.   A  detailed  critique  should  also  be 
conducted  for  all  involved  in  the  evaluation.   In  the  critique 
the  results  of  the  evaluated  MPS  are  to  be  highlighted  with 
respective  strengths  and  weaknesses. 

3 .  Evaluators 

Evaluators  should  be  prepared  for  the  role  through 
successful  past  professional  experiences  as  well  as  through 
any  detailed  school  provided.   The  MCCRES  evaluators  have 
three  roles:   exercise  controller,  umpire,  and  performance 
evaluator.   As  a  performance  evaluator,  they  apply  the  de- 
tailed MPS  '  s  contained  in  the  appropriate  volume  for  the  unit 
and  they  are  evaluating  and  actually  make  the  'Yes/No/Not 
Applicable'  determinations  for  individual  tasks.   The  evalua- 
tors must  possess  a  complete  and  thorough  understanding  of 
the  Mission  Performance  Standards  being  evaluated.   The  evalu- 
ators must  make  any  notes  needed  on  an  Evaluator  Work  Sheet 
that  are  necessary  to  support  judgment  during  the  process 
[Ref .  IJ . 

4 .  Senior  Evaluator 

Senior  Evaluators  will  be  determined  by  rank.  Evalu- 
ator for  the  unit  evaluated  compiles  the  data  sheets  from  all 
evaluators  and  should  conduct  a  post  exercise  wrap  up.   Any 
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questions  or  conflicts  should  be  resolved  at  that  time.   The 
process  must  result  in  the  senior  evaluator  being  assured  that 
his  data  provides  an  accurate  reflection  of  the  overall  unit 
performance.   The  senior  evaluator  then  makes  the  determina- 
tion of  the  overall  unit  evaluation  of  "Combat  Ready/Not 
Combat  Ready".   The  senior  evaluator  provides  the  data  sheets 
to  TEC  who  compiles  the  detailed  analysis  and  presents  it  to 
the  evaluation/exercise  director  [Ref.  1]. 
5 .   Evaluation  Staffing 

Evaluation  Staffing  is  the  responsibility  of  the  ED 
for  both  selecting  and  training  all  evaluators  prior  to  a 
MCCRES .   It  is  desirable  to  obtain  evaluators  from  adjacent  com- 
mands, not  directly  related  to  the  organization  being  evaluated, 

D.   COMPUTER  ADAPTED  MCCRES 

The  Marine  Corps  Combat  Readiness  System  Software  Applica- 
tion (MCCRESSA) ,  is  designed  to  furnish  all  organizations  in- 
volved with  MCCRES  a  means  of  assessing  and  analyzing  data 
pertaining  to  MCCRES  evaluations.   Specifically  it  provides 
the  following: 

A  list  of  mission  performance  standards,  tasks,  and 
requirements . 

...the  chief  evaluator  with  a  rapid  overview  and  selected 
analysis  of  the  units  evaluation. 

Identifies  unit  readiness  based  upon  mission  performance 
standards  tested  within  the  applied  scenario. 

Provides  unit  commanders  a  list  of  unit  readiness  deficien- 
cies to  initiate  corrective  action  or  obtain  higher  command 
level  assistance. 
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Reviews  unit  evaluations  to  determine  validity  of  mission 
performance  standards  and  doctrine. 

Reviews  unit's  evaluations  to  determine  limitation/inade- 
quacies in  officer  education  programs.   [Ref.  1:  p.  I-E2] 

All  of  the  formal  evaluation  reports  are  input  to  the  Head- 
quarters Marine  Corps  data  base  [Ref.  1] . 

E.   REPORTS 

At  the  end  of  a  MCCRES ,  after  the  Evaluator  Data  Sheets 
have  been  compiled  for  all  units  evaluated,  the  TEC  and  senior 
evaluators  should  review  the  sheets  and  make  a  determination 
of  "Combat  Ready /Not  Combat  Ready"  for  each  unit  evaluated. 
This  recommendation  is  based  on  an  initial  review  of  "Yes's/ 
No • s" f or  all  requirements,  taking  into  account  any  tasks  or 
MPS ' s  that  were  "demand  elements  that  must  be  judged  Yes  for 
an  overall  'Combat  Ready*  evaluation  to  be  given." 

1 .   Feedback 

"The  primary  purpose  of  MCCRES  reports  is  to  provide 
the  feedback  necessary  for  commanders  to  initiate  corrective 
action  that  will  im.prove  combat  readiness"  [Ref.  1:  p.  1-DlJ  . 
The  results  should  be  reflected  in  improved  training 
objectives  and  may  also  affect  resource  allocation  of  per- 
sonnel, equipment,  or  logistics  support.   To  assist  in  the 
reallocation  of  critical  resources,  a  detailed  report  is  pro- 
vided through  the  chain  of  command  to  the  unit  evaluated. 
This  report  provides  a  short  subjective  comment  as  well  as 
percentile  scores  and  weighting  for  each  Section,  MPS,  Task, 
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and  Requirement  evaluated.   (See  Appendix  A — Sample  MCCRESSA 
printout  for  one  MPS . ) 

Additionally  a  report  is  provided  in  message  format 
to  the  Commandant  of  the  Marine  Corps  (CMC)  within  10  days 
following  the  end  of  an  evaluation  indicating  the  overall 
evaluation  "Combat  Ready/Not.  Combat  Ready"  and  unit  identifi- 
cation.  Within  30  days  a  detailed  report  of  the  MCCRESSA 
printout  is  provided  to  the  CMC.   The  report  should  include 
comments  or  recommendations  for  improvement  or  revisions  to 
the  MCCRES.   The  purpose  of  the  follow  up  report  is  to  allow 
CMC  to;   provide  assistance  in  the  review  of  doctrine,  tactics, 
techniques,  education  and  training  programs,  and  validation 
of  MCCRES  elements.   This  report  also  serves  to  highlight 
trends  and  repeated  deficiencies,  and  permit  analysis  for 
corrective  action  to  take  place  [Ref .  Ij  . 
2 .   Policy 

Based  on  Marine  Corps  policy,  MCCRES  reports  are  not 
designed  to  be  used  to  compare  the  combat  readiness  of  various 
units.   This  is  a  result  of  differences  in  unit  type,  services, 
environmental  conditions,  and  resource  allocations.   The 
evaluation  indexes  on  MCCRES  reports  are  provided  to  indicate 
an  approximate  status  of  the  unit's  combat  readiness  during 
a  particular  evaluation  [Ref.  1].   It  can  be  said  that  the 
MCCRES  provides  a  snap  shot  of  the  unit  at  the  time  of  the 
evaluation. 
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F .   SUMMARY 

This  chapter  presented  an  overview  of  the  MCCRES  including 
a  discussion  of  the  Development  and  Structure,  Mission  Per- 
formance Standards,  Personnel  Assignment  for  the  Evaluation, 
Computer  Adapted  MCCRES  and  Reports .   The  purpose  was  to 
provide  the  reader  with  a  general  background  of  how  the 
MCCRES  is  intended  to  be  implemented  throughout  the  Marine 
Corps,  based  on  systems  design  and  policies. 
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V.   METHOD  OF  ANALYSIS 

The  research  method  used  had  three  distinct  phases.   First 
a  general  literature  review  was  conducted.   Second  a  specific 
literature  review  of  MCCRES  and  related  defense  publications 
was  undertaken.   Third  a  detailed  field  study  was  made  to 
obtain  first  hand  information  from  those  involved  with  the 
development,  application,  and  oversite  of  MCCRES. 

A.  AUDIENCE 

Anderson  and  Ball  specified  in  their  findings  [Ref.  36] 
of  the  importance  of  identifying  the  audience  when  researching 
and  distributing  results.   To  this  end  the  primary  audience 
for  this  study  are  the  planners  who  conduct  periodic  reviews 
and  revisions  of  MCCRES  doctrine,  that  is,  the  Readiness  Branch 
at  HQMC.   Additionally  the  audience  should  include  all  those 
involved  with  MCCRES  or  who  will  be  involved  with  MCCRES,  as 
well  as  commanders,  evaluators,  planners  in  organizations  and 
others  who  would  benefit  from  understanding  the  findings  pre- 
sented herein. 

B.  GENERAL  LITERATURE  REVIEW 

The  results  of  the  general  literature  review  are  presented 
in  Chapter  III.   A  background  in  management  control,  general 
evaluation  systems  as  well  as  specific  examples  applied  to 
the  military  from  various  disciplines  is  necessary  to  gain 
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insights  into  MCCRES .   It  is  evident  from  Chapter  IV  that 

MCCRES  has  incorporated  into  it  many  of  the  positive  aspects 

of  general  evaluation  systems  theory  [Refs.  5,6,20,25,26, 
32,33] . 

C.  SPECIFIC  LITERATURE  REVIEW 

The  specific  literature  review  consisted  of  articles  on 
MCCRES  appearing  in  the  various  defense  publications  [Refs.  2, 
6,10,13,31,37,  and  38]  as  well  as  the  technical  reports  and 
articles  available  on  MCCRES  and  other  military  training  and 
readiness  evaluation  systems  [Refs.  4,7,11,12,17,38].   This 
specific  literature  review  not  only  provided  a  basis  for 
understanding  the  MCCRES  from  the  standpoint  of  its  proponents 
and  detractors,  but  it  also  provided  some  valuable  ideas  on 
alternatives  to  MCCRES.   The  detailed  technical  reports  made 
available  by  George  Washington  University  were  invaluable  in 
understanding  the  background  and  initial  development  of  the 
MCCRES  [Refs.  3,4,10] . 

D.  FIELD  STUDY 

The  field  study  was  conducted  to  obtain  first  hand  infor- 
mation on  MCCRES.   Personal  interviews,  telephone  interviews, 
and  a  visit  to  the  Readiness  Branch  HQMC,  George  Washington 
University,  and  to  the  Army's  7th  Infantry  Division  were  em- 
ployed to  that  end.   The  interviews  were  semi-structured.   The 
stated  purpose  of  the  interviews  was  not  just  to  answer  the 
specific  questions  posed,  but  to  stimulate  thought  and  identify 
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as  many  ideas  as  possible  on  how  the  MCCPIES  could  be  improved. 
Although  the  scope  of  this  project  was  limited  by  its  primary 
emphasis  on  feedback,  other  valuable  information  was  elicited. 
1.   Sample 

The  sample  group  chosen  consisted  of  those  who  had 
the  most  experience  and  most  vested  interest  in  the  MCCRES 
feedback.   Because  the  MCCRES  evaluations  are  most  often 
given  to  infantry  battalions  and  aviation  squadrons,  these 
were  identified  as  the  primary  target  group.   Battalion  and 
squadron  commanders,  senior  evaluators,  and  division/squadron 
MCCRES  officers  were  identified  as  the  primary  sources  of 
information  since  they  have  the  most  detailed  involvement  with 
the  MCCRES  feedback.   Upon  interviewing  the  sample  it  was  also 
found  that  many  of  the  officers  had  been  involved  in  several 
MCCRES ' s  and  in  different  capacities.   Thus  many  had  experi- 
ence as  both  a  senior  evaluator  and  a  battalion  or  squadron 
commander  of  an  evaluated  unit.   Divisional  or  squadron  MCCRES 
officers  were  also  a  rich  source  of  information  because  of 
their  daily  involvement  with  MCCRES.   The  secondary  source  of 
information  were  "others  involved  with  MCCRES."   This  group 
is  composed  of  members  of  evaluated  units  other  than  the 
unit  commanders,  such  as  unit  operations  officer,  officers  in- 
volved with  subunits  evaluated,  and  company  commanders.   They 
also  provided  much  information. 

Given  the  sample  identified,  the  field  study,  there- 
fore, concentrated  on  those  of  rank  from  captain  to  colonel 
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in  the  military  occupational  f  ielc3s  of  infantry,  aviation 
(fixed  wing  and  helicopters),  tanks,  artillery,  engineer, 
and  air  control.   A  total  of  37  individuals  were  interviewed, 
The  sample  group  is  presented  by  rank  and  occupational  field 
below  in  Table  5.1. 

TABLE  5.1 
Rank/Occupational  Field  Sample  Distribution 
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Colonel  3  10  0     0  0  0 

Lt.  Colonel  10  3     3  0     1  •  1  1 

Major  3  4      3  10  0  0 

Captain  2  0     0  1     0  '  0  0 


TOTAL: 


18 


4 

19 

11 

3 

37 


These  officers  are  currently  serving  in  billets  throughout 
the  Marine  Corps,  representing  all  divisions  and  wings,  as 
well  as  the  1st  Marine  Brigade  in  Hawaii.   Other  active  duty 
officers  serving  in  billets  at  Headquarters  Marine  Corps, 
Naval  Postgraduate  School,  and  with  the  reserve  establishment 
were  also  interviewed. 

2 ,   Interview  Techniques 

All  those  interviewed  were  asked  a  standard  set  of 
open-ended  questions  [Ref.  45].   These  questions  were 


87 


purposefully  left  broad  and  open-ended  to  solicit  as  many 
ideas  and  opinions  as  possible.   Each  interview  was  conducted 
in  person  or  by  telephone  by  this  researcher.   All  interview 
results  were  recorded  on  an  answer  form  and  interviewers  were 
taped  after  the  permission  of  the  respondent  was  given.   When 
a  respondent  was  uncertain  or  uncomfortable  in  answering  a 
question,  he  was  asked  to  skip  over  it.   However,  no  attempt 
was  made  to  limit  the  respondent  to  any  set  of  prearranged 
answers.   In  all  cases,  where  answer  choices  were  posed, 
an  invitation  for  "other  ideas  or  suggestions"  was 
always  made.   As  the  interviews  progressed,  the  questions  were 
refined  to  help  the  interviewee  narrow  in  on  the  specific 
area  being  researched. 
3.   Questions  Posed 

The  following  set  of  questions  was  posed  to  all  those 
interviewed: 

(1)  When  the  computerized  data  results  from  a  MCCRES  are 
provided  to  the  evaluated  unit,  what  uses  are  made  of 
that  data? 

(2)  Do  you  feel  that  information/data  provided  to  the 
evaluated  unit  as  the  result  of  a  MCCRES  are  satisfac- 
tory for  the  purpose  of  initiating  corrective  action 
that  will  improve  combat  readiness? 

(3)  Is  the  current  procedure  of  providing  the  results  as 
a  computer  printout  based  on  the  evaluation  of  the 
Sections,  MPS,  Tasks,  and  Requirements  the  most 
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effective  format  for  displaying  the  results  to  the 
evaluated  unit  commander? 

(4)  Does  the  format  lend  itself  to  a  complete  understanding 
of  how  the  unit  performed  compared  with  MCCRES  criteria? 

(5)  Do  Marine  Corps  Directives  provide  sufficient  guidance 
for  interpretation  of  results? 

(6)  Would  it  be  helpful  to  know  how  your  unit  performed  on 
a  Section/MPS/Task/Requirement  as  compared  with  other 
units?   Example:   Your  infantry  battalion  scored  a 

73%  on  the  MPS,  "Command  and  Control  Operations." 

This  compares  with  a  median  of  68%  for  all  other  infantry 

units  and  71%  for  all  units  evaluated. 

(7)  What  do  you  feel  is  the  most  effective  forum  for  pro- 
viding MCCRES  results  to  the  unit  tested? 

a.  Oral  debrief  at  end  of  .exercise 

b.  Written  debrief  at  end  of  exercise 

c.  Progressive  oral  debrief  throughout  the  exercise 

d.  Combination  of  a-c  above 

e.  Some  other  method  than  those  suggested 

(8)  Who  should  be  made  aware  of  the  MCCRES  results? 

a.  Chain  of  command 

b.  Other  like-units  (infantry,  air,  artillery,  etc.) 

c.  All  Marine  Corps  units 
4 .   Background  Visits 

Additionally  visits  were  made  to  Headquarters  Marine 
Corps  (HQMC) ,  Readiness  Branch,  George  Washington  University 
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and  to  the  Army's  7th  Infantry  Division,  Fort  Ord,  California. 
Individuals  interviewed  during  these  visits  provided  many 
valuable  insights  into  the  development  and  overseeing  of 
MCCRES,  as  well  as  a  detailed  understanding  of  how  the  Army 
ARTEP  system  is  employed.   Several  of  the  individuals  inter- 
viewed were  involved  in  the  early  development  of  the  MCCRES : 
Professor  W.  H.  Marlowe,  George  Washington  University;  Lt . 
Colonel  Paul  Catalone,  USMC,  and  Colonel  M.  P.  Sullivan,  USMC 
(telephone  interview),  and  Lt.  Colonel  R.  S.  Gibson,  USMC 
(telephone  interview) . 

E.   CONTENT  ANALYSIS/DATA  REVIEW 

Since  the  material/responses  to  the  questions  asked  were 
subjective  in  nature,  content  analysis  was  used  to  analyze 
the  results  [Ref.  46]. 

1.   Content  Analysis  Defined 

Berelson  states:   "Systematic  content  analysis  attempts 
to  define  casual  descriptions  of  the  content,  so  as  to  show 
objectively  the  nature  and  relative  strength  of  the  stimuli 
applied  to  the  reader  or  listener"  [Ref.  46:  p.  14]. 

This  procedure  is  employed  by  constructing  an  analysis 
outline  which  described  by  Cartwright  embodies  the  following 
general  principles: 

Step  1 — specify  data  needs, 

Step  2 — map  out  plans  for  tabulation. 

Step  3 — map  a  skeleton  of  the  outline. 

Step  4--fill  in  categories  for  each  variable. 
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step  5 — establish  procedures  for  using  the  material. 

Step  6 — try  out  the  analysis  outline  and  use  procedure- 
[Ref.  45:  pp.  454-460] 

The  application  of  content  analysis  allows  the  researcher 
to  study  the  communications  and  focus  on  the  interaction  through 
messages  which  link  communicating  parties.   Not  all  signifi- 
cance can  be  extracted  by  inspection  or  mere  observation. 
The  real  purpose  of  any  analysis  is  to  illuminate  and  make 
inferences  about  something  that  is  not  otherwise  apparent. 
Thus  the  process  of  content  analysis  is  a  process  like  the 
congruence  definition  of  evaluation  which  employs  objectives 
and  criteria  in  developing  a  measurement  process  which  is 
both  scientific  and  objective.   The  process  is  not,  however, 
totally  devoid  of  judgment  and  does  allow  the  researcher  dis- 
cretion in  categorizing  results  [Ref .  47] . 

2 .   Analysis  of  Specific  Responses  to  Questions 

The  purpose  of  content  analysis  is  to  provide  some 
means  of  quantifying  subjective  information.   To  accomplish 
this  rules  must  be  established.   The  data  contained  in  the 
response  to  each  of  the  eight  questions  was  treated  as  a 
separate  unit  and  no  attempt  was  made  during  the  interview  to 
relate  any  answers  from  one  question  to  that  of  another.   This 
procedure  was  used  to  solicit  as  much  information  possible  and 
so  as  not  to  limit  any  of  the  respondents  in  their  comments. 
Since  each  question  stood  alone  and  was  relatively  subjective 
in  nature,  each  question  provided  an  opportunity  for  a  response 
to  contain  similar  information  in  response  to  different 
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questions.   Thus  once  a  respondent  began  to  provide  information 
to  a  question  which  was  more  appropriately  categorized  as  a 
response  to  a  different  question,  the  interviewer  simply  re- 
corded the  information  under  the  question  currently  being 
asked.   However  as  part  of  the  analysis  of  the  interviews, 
responses  to  one  question  that  were  relevant  to  other  questions 
were  considered. 

Each  question  had  certain  categories  set  up  for  the 
responses  and  each  response  made  was  placed  in  one  of  these 
categories.   In  order  to  generate  useful  information  from  con- 
tent analysis,  the  categories  must  be  collectively  exhaustive 
and  mutually  exclusive.   For  example: 

Question  #1 — When  the  computerized  data  results  from  a  MCCRES 
are  provided  to  the  evaluated  unit,  what  uses  are  made  of 
that  data? 
Responses  to  this  question  fell  into  three  groupings: 

"none" 

"used  for  planning  future  training  and  resource  allocation" 

"historical  reference  only" 

It  should  be  understood  that  each  MCCRES  has  many  key 
participants  and  each  has  a  different  involvement  in  the 
evaluation.   Some  such  as  the  unit  commander  and  senior  evalu- 
ator  are  involved  virtually  in  all  aspects.   Others  such  as 
the  operations  officers  and  operations  evaluators  have  a  more 
limited  focus  in  their  responsibilities.   Thus  a  system  was 
developed  to  give  varying  weight  to  those  who  had  differing 
levels  of  involvement  with  MCCRES. 
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A  scoring  system  was  established  with  the  following 

weighting: 

Intensity  Points 

High  3 

Medium  2 

Low  1 

High  Intensity — those  involved  directly  in  one  or  more 

MCCRES  as  unit  commander,  division  or 
wing  MCCRES  officer,  senior  evaluator. 

Medium  Intensity — those  involved  with  less  direct 

responsibility  for  results:   operations 
officers,  other  evaluators,  company 
commanders,  etc. 

Low  Intensity — Employed  in  only  one  case  where  interviewee 

was  involved  in  a  forerunner  evaluation  to 
MCCRES 

3 .   Analysis  of  General  Responses  to  Questions 

To  determine  which  question  responses  should  be  dis- 
played for  the  reader,  a  10%  rule  was  formulated.   If  10%  or 
more  of  the  respondents  to  a  question  amplified  their  specific 
answer  with  similar  comments  or  suggestions,  then  the  comments 
were  included.   Thus,  any  of  the  general  responses  included 
in  the  next  chapter  were  mentioned  by  four  or  more  of  those 
sampled. 

F.   SUMMARY 

In  this  chapter  the  research  method  is  summarized.   A 
total  of  3  7  Marine  officers  were  interviewed.   Although  the 
number  is  relatively  small,  it  should  be  remembered  that  the 
Marine  Corps  has  three  infantry  divisions  and  three  aircraft 
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wings  on  active  duty.   Therefore,  the  total  population  of  those 
qualified  to  answer  the  questions  is  drawn  from  a  limited 
population  of  officers  in  the  target  group.   It  is  hard  to 
estimate  the  total  number  of  officers  that  have  been  involved 
in  these  primary  capacities  since  the  billet  holders  rotate 
regularly;  but  it  is  safe  to  say  that  only  a  few  hundred  offi- 
cers have  served  as  battalion/squadron  couimanders  or  senior 
evaluators  of  evaluated  units.   Since  the  MCCRES  is  only  re- 
quired to  be  given  every  two  years,  the  sample  used  appears 
to  be  reasonable. 
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VI.   RESULTS 

This  chapter  presents  the  detailed  analysis  of  the  results 
obtained  through  the  field  study.   The  specific  and  general 
responses  to  questions  posed  were  content  analyzed.   The  basis 
for  this  type  of  analysis  is  provided  by  Berelson  [Ref .  46] . 

Percentages  derived  from  the  content  analysis  are  the 
weighted  averages  of  the  points  for  each  category  as  compared 
with  total  points  for  each  question.   The  responses  are  weighted 
based  upon  the  level  of  involvement  of  the  respondent  in  previ- 
ous MCCRES ' s .   In  cases  where  some  interviewees  did  not  answer 
a  certain  question,  the  percentage  is  based  only  on  the  re- 
sults of  those  who  did  answer.   No  attempt  was  made  to  cate- 
gorize reasons  why  those  that  failed  to  respond  did  so. 

A.   ANALYSIS  OF  SPECIFIC  RESPONSES  TO  QUESTIONS 

First  Question:   (1)   When  the  computerized  data  results 

from  a  MCCRES  are  provided  to  the  evaluated  unit,  what  uses 

are  made  of  that  data? 

Responses  to  this  question  were  grouped  into  three  categories- 

None,  Used  for  planning,  and  Historical  reference  only — and  are 

shown  in  Table  6.1.   The  majority  of  the  respondents  (63.6%). 

said  that  the  results  of  MCCRES  were  used  for  planning  future 

training  or  specifically  for  future  resource  allocation. 

Second  Questions:   (2)   Do  you  feel  that  information/data 

provided  to  the  evaluated  unit  as  a  result  of  MCCRES  is 
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satisfactory  for  the  purpose  of  initiating  corrective  action 
that  will  improve  combat  readiness? 

The  responses  to  this  question  are  categorized  as  positive 
or  negative  and  are  shown  in  Table  6.2.   The  majority  of  the 
respondents  (81.3%)  said  that  the  information/data  provided 
from  a  MCCRES  is  satisfactory  for  initiating  corrective  action 
that  will  improve  combat  readiness. 

Third  Question:   (3)   Is  the  current  procedure  of  provid- 
ing the  results  as  a  computer  printout  based  on  the  results 
of  Sections,  MPS,  Tasks,  and  Requirements,  the  most  effective 
format  for  displaying  the  results  to  the  evaluated  unit? 

The  responses  to  this  question  were  categorized  as  posi- 
tive or  negative.   As  shown  by  Table  6.3,  64.6%  of  the 
respondents  said  that  the  current  procedure  of  providing  the 
results  as  a  computer  printout  was  not  the  most  effective 
format  for  displaying  the  results  to  the  evaluated  unit. 

Fourth  Question:   (4)   Does  the  format  lend  itself  to  a 
complete  understanding  of  how  the  unit  performed  compared  with 
MCCRES  criteria? 

The  response  to  this  question  was  categorized  as  positive 
or  negative.   As  shown  in  Table  6.4,  5  5.6%  of  the  respondents 
said  that  the  current  format  lends  itself  to  a  complete  under- 
standing of  how  the  unit  compared  with  MCCRES  criteria. 

Fifth  Question:   (5)   Do  Marine  Corps  Directives  provide 
sufficient  guidance  for  interpretation  of  results? 

The  responses  to  this  question  were  categorized  as  posi- 
tive or  negative.   As  shown  by  Table  6.5,  72.9%  of  the 
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respondents  said  that  Marine  Corps  Directives  provide  suffi- 
cient guidance  for  interpretation  of  results. 

Sixth  Question:   (6)   Would  it  be  helpful  to  know  how  your 
unit  performed  on  a  Section/MPS/Task/Requirement  as  compared 
with  other  units?   Example:   Your  infantry  battalion  scored 
a  73%  on  the  MPS,  "Command  and  Control  Operations."   This 
compares  with  a  median  of  68%  for  all  other  infantry  units 
and  71%  for  all  units  tested. 

The  responses  to  this  question  were  categorized  as  posi- 
tive or  negative.   As  shown  by  Table  6.6,  55.4%  of  the  respondents 
said  that  it  would  be  helpful  to  know  how  their  unit  performed 
on  a  Section/MPS/Task/Requirement  as  compared  with  other  units. 

Seventh  Question:   (7)   What  do  you  feel  is  the  most 
effective  forum  for  providing  MCCRES  results  to  the  unit? 

This  question  was  evaluated  based  on  selection  of  one  of 
five  responses:  oral  debrief  at  end  of  exercise,  written  at 
end,  progressive;  combination  of  above;  or  other.  As  indicated 
in  Table  6.7,  all  thirty-seven  interviewees  responded  to  the 
question.  Of  the  total,  91.5%  responded  that  a  combination 
of  oral  and  written  debriefs  is  the  most  effective  forum  for 
providing  MCCRES  results  to  the  unit  evaluated. 

Eighth  Question:   (8)   Who  should  be  made  aware  of  MCCRES 
results? 

The  question  was  evaluated  based  on  selection  of  one  of 
three  responses:   chain  of  command,  other  like  units,  or  all 
units.   As  indicated  in  Table  6.8,  60.2%  of  the  respondents 
indicated  that  all  units  should  be  made  aware  of  MCCRES  results. 
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The  responses  to  the  questions  are  summarized  for  the 
reader  in  Tables  6.9  and  6.10.   Table  6.9  shows  the  distribu- 
tion by  rank  of  the  specific  responses  to  the  eight  questions. 
Table  6.10  shows  the  distribution  by  community  (ground/air). 
A  more  detailed  display  of  data  by  occupational  specialty  was 
available  to  the  researcher;  however,  to  protect  the  anony- 
mity of  the  respondents  only  the  general  categorizations  of 
answers  is  displayed.   In  reviewing  the  data  in  Tables  6.9 
and  6.10  there  is  no  direct  evidence  of  bias  by  any  specific 
rank  or  community.   This  same  result  was  true  when  evaluating 
the  data  by  occupational  specialty. 

B.   ANALYSIS  OF  GENERAL  RESPONSES  TO  QUESTIONS 

The  primary  purpose  of  this  research  was  to  gain  insight 
and  new  ideas.   To  this  end  the  respondents  were  specifically 
asked  to  expand  upon  and  explain  their  answers.   Their  explana- 
tions resulted  in  the  following  information.   The  10%  decision 
rule  discussed  above  was  used  to  determine  which  information 
to  present. 

In  response  to  Question  One,  the  majority  of  the  inter- 
viewees said  that  the  results  from  a  MCCRES  found  their  way 
into  unit  training  plans  and  schedules.   One  of  the  most  serious 
concerns  brought  out  in  response  to  this  question  concerned 
timing  [Ref.  20]  of  the  evaluation.   If  the  MCCRES  is  given 
sixty  days  or  more  prior  to  deployment  then  the  unit  may 
experience  substantial  turnover  of  key  personnel  and  nullify 
the  results  of  the  MCCRES  by  changing  key  players.   The 
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TABLE    6.9 
Distribution  by   Rank  of   Responses    to   Questions 
Question  #  Colonel     Lt.   Colonel     Major     Captain         TOTAL 


1. 

a. 

None 

0 

4 

1 

0 

5 

b. 

Planning 

4 

11 

4 

3 

22 

c. 

Historical 

0 

3 

4 

0 

7 

2. 

a. 

Positive 

3 

16 

7 

3 

29 

b. 

Negative 

0 

4 

2 

0 

6 

3. 

a. 

Positive 

1 

7 

2 

1 

U 

b. 

Negative 

2 

11 

6 

0 

19 

4. 

a. 

Positive 

3 

8 

3 

1 

15 

b. 

Negative 

0 

9 

3 

0 

12 

5. 

a. 

Positive 

2 

12 

6 

0 

20 

b. 

Negative 

0 

5 

1 

1 

7 

6. 

a. 

Positive 

3 

10 

7 

1 

21 

b. 

Negative 

1 

9 

4 

1 

15 

7. 

a. 

Oral 

0 

0 

0 

0 

0 

b. 

Written 

0 

1 

0 

0 

1 

c. 

Progressive 

0 

2 

0 

0 

2 

d. 

Combination 

4 

17 

10 

3 

34 

e. 

Other 

0 

0 

0 

0 

0 

8. 

a. 

Chain 

1 

7 

2 

1 

11 

b. 

Like 

0 

1 

2 

1 

4 

c. 

All 

3 

10 

7 

1 

21 
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TABLE  6.10 
Distribution  by  Community  of  Responses  to  Questions 

Aviation  TOTAL 

1  5 

7  22 
5  7 

8  29 

5  6 

4  11 

8  19 

6  15 
6  12 

8  20 
4  7 

9  21 

6  15 

0  0 

0  1 

0  2 

15  34 

0  0 

2  11 
2  4 

10  21 

KEY:   Ground:  Infantry,  Tanks,  Artillery,  Engineer 

Aviation:   Fixed  wing.  Helicopter,  Air  Control 


Quest 

ion 

_i 

Ground 

1. 

a . 

None 

4 

b. 

Planning 

15 

c. 

Historical 

2 

2. 

a. 

Positive 

21 

b. 

Negative 

1 

3. 

a. 

Positive 

7 

b. 

Negative 

11 

4. 

a. 

Positive 

9 

b. 

Negative 

6 

5. 

a. 

Positive 

12 

b. 

Negative 

3 

6. 

a. 

Positive 

12' 

b. 

Negative 

9 

7. 

a. 

Oral 

0 

b. 

Written 

1 

c. 

Progressive 

2 

d. 

Combination 

19 

e. 

Other 

0 

8. 

a. 

Chain 

9 

b. 

Like 

2 

c. 

All 

11 
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alternative  of  this  presents  a  serious  dileirana.   If  the  MCCRES 
is  given  too  close  to  time  of  deployment,  then  little  time 
exists  to  correct  all  but  the  most  glaring  deficiencies. 
Additionally  since  no  real  second  evaluation  is  made  of  the 
shortcomings,  the  USMC  may  be  deploying  units  who  have  not 
in  fact  corrected  shortcomings  identified  during  MCCRES. 

A  second  major  problem  identified  by  the  respondents  is 
the  timeliness  with  which  the  unit  gets  the  actual  MCCRES  re- 
port.  Some  organizations  present  at  least  a  rough  of  the 
printout  to  the  unit  at  the  time  of  the  formal  debrief,  usually 
12-24  hours  after  termination  of  the  MCCRES.   Others  do  not 
receive  their  formal  printout  until  there  is  a  complete  staffing 
by  division/wing,  regiment/group;  and  in  at  least  two  cases 
the  unit  did  not  receive  their  formal  feedback  until  they  were 
on  their  deployment.   One  unit  did  not  see  their  printout  at 
all  after  the  MCCRES.   (This  comment  is  provided  as  an  exception 
to  the  10%  rule  because  of  the  seriousness  of  this  situation.) 

Responses  to  Question  Two  indicated  that  the  current 
MCCRES  gives  sufficient  information  to  improve  combat  readi- 
ness.  In  the  case  of  the  rotoiywing  MCCRES,  three  of  the  six 
rotorywing  respondents  indicated  that  more  weight  was  needed 
on  flying  skills  than  is  currently  given  by  the  MPS  in  Volume 
III.   Generally  the  responses  from  the  rotor  wing  community 
indicated  too  much  emphasis  is  given  to  non-flight  requirements 
and  too  little  to  flight  skills.   In  addition  responses  from 
the  fixed-wing  community  indicated  that  technology  is  changing 
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more  rapidly  than  the  MCCRES .   With  the  implementation  of  the 
FA-18  aircraft  and  changes  to  other  aircraft,  the  MCCRES 
evaluation  needs  to  be  changed  to  be  current  and  realistic. 
In  other  words  the  aircraft  can  do  a  lot  more  than  MCCRES  re- 
quires.  Discussions  with  the  Readiness  Branch  indicate  they 
are  well  aware  of  the  problems  caused  by  changing  technology 
in  the  FA-18  and  other  new  aircraft,  and  are  presently  improv- 
ing the  MCCRES  requirements  to  test  the  aircraft  at  limits 
that  more  closely  approximate  its  actual  capabilities  rather 
than  against  the  less  demanding  F-4  Requirements. 

A  total  of  13  respondents  commented  that  MCCRES  results  are 
only  as  credible  as  the  evaluators.   The  absence  of  standardi- 
zation of  evaluators  and  evaluator  training,  as  well  as  the 
variances  in  how  organizations  pick  evaluators  had  a  great 
impact  on  the  value  perceptions  and  the  worth  of  the  MCCRES 
as  a  measure  of  the  unit's  readiness.   This  was  expressed  by 
six  of  the  fifteen  officers  in  the  aviation  community.   Forty 
percent  of  the  respondents  submitted  an  unsolicited  comment  on 
the  quality  of  the  evaluators.   If  the  evaluator  is  not  re- 
garded as  highly  qualified  by  his  peers,  then  the  evaluation 
is  regarded  with  little  weight.   The  question  of  evaluator 
bias  is  discussed  in  more  depth  by  Wheeler  [Ref.  5]. 

A  final  problem  cited  dealt  with  the  MCCRES  concept  of 
comparing  unit  performance  with  published  combat  doctrine. 
Doctrine  as  codified  in  written  manuals  may  be  outdated.   If 
so,  the  USMC  is  evaluating  units  by  comparing  their  performance 
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to  standards  which  would  not  be  employed  on  the  modern  battle- 
field.  An  example  af  this  cited  by  four  infantry  commanders 
is  "maneuver  warfare".   Maneuver  warfare  is  widely  accepted 
by  Western  military  analysts  as  the  most  promising  way  to 
fight  and  win  any  future  war  against  Soviet  tactics.   However, 
maneuver  warfare  is  not  written  into  current  Marine  Corps 
doctrinal  publications.   This  shortfall  points  out  the  need 
for  distribution  of  MCCRES  results  to  all  training  and  doctrine 
commands  to  ensure  they  are  aware  of  the  strengths  and  short- 
falls that  units/individuals  display  on  MCCRES. 

The  most  common  response  to  Question  Three  was  negative. 
A  percentage  of  64.6%  of  the  respondents  said  that  they  did 
not  feel  the  computerized  printout  is  the  best  way  to  dis- 
play MCCRES  results.   Nineteen  of  thirty  respondents  indicated 
more  subjective  data  was  needed  to  fully  understand  the  units' 
evaluations  and  to  be  able  to  take  corrective  actions.   The 
computer  printout  provides  a  "Yes/No"  evaluation,  but  it  fails 
to  provide  the  "why"  and  "what"  [Ref .  36] .   More  details  are 
needed  to  describe  specific  reasons  for  "No"  marks  and  recom- 
mendations are  necessary  to  help  the  unit  find  ways  to  improve. 
All  those  responding  negatively  C19  of  the  30  respondents) 
indicated  it  was  much  more  valuable  to  spell  out  the  details 
of  a  shortfall  and  to  suggest  corrective  action,  than  to  simply 
provide  a  "Yes/No"  on  an  evaluation  sheet.   Several  innovative 
ideas  have  come  out  of  this  problem.   One  wing  MCCRES  officer 
now  writes  all  comments  directly  on  the  MCCRES  printout  before 
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it  is  provided  to  the  unit.   The  wing  MCCRES  officer  said 
this  facilitates  a  much  more  complete  understanding  of  problem 
areas.   The  general  concensus  of  the  30  respondents  to  this 
question  was  that  the  subjective,  explanatory  information  is 
by  far  the  most  valuable  to  the  evaluated  unit  and  the  per- 
centile scores  on  the  printout  are  probably  of  greater  value 
to  higher  level  headquarters  for  use  in  planning  future  training. 

In  response  to  Question  Four,  15  of  the  27  respondents  or 
55.6%  agreed  that  the  current  format  is  not  difficult  to  under- 
stand.  In  general  those  who  indicated  a  willingness  to  refer- 
ence the'  directives  and  compare  the  printout  to  the  appropri- 
ate Volume  were  satisfied  with  the  explanations  provided 
therein.   Those  who  said  the  current  format  is  difficult  to 
understand  (12  of  the  27  or  44.4%)  did  so  based  on  a  criterion 
of  the  inconvenience  of  the  system,  that  is  requiring  both 
the  printout  and  the  Volume.   This  requires  a  good  bit  of 
tedious  research  to  find  the  details  of  each  MPS/Task/Require- 
ment.   The  recent  addition  of  a  description  of  each  MPS/Task/ 
Requirement  on  the  MCCRES  printout  should  help  reduce  this 
problem.   On  the  other  hand,  the  brief  description  may  cause 
even  less  referencing  of  the  source  document  to  obtain  complete 
details.   One  suggestion  was  that  key  players  from  the  unit  to  be 
evaluated  should  be  given  detailed  classes  and  instructions  on 
the  MPS  several  weeks  prior  to  a  scheduled  MCCRES.   This  has 
recently  been  started  in  one  wing  and  the  results  are  excellent. 
This  contention  was  supported  by  both  squadron  commanders 
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questioned  as  well  as  by  those  in  the  Readiness  Branch  at 
Headquarters  Marine  Corps.   Just  as  there  is  a  need  to  edu- 
cate evaluators,  there  is  also  a  need  to  educate  those  to  be 
evaluated  [Refs.  5,25,26]. 

An  added  point  made  by  five  unit  commanders  was  that  the 
results  must  get  out  to  all  those  concerned.   If  the  response 
to  an  evaluation  is  delayed  or  the  results  simply  become  his- 
torical record,  those  who  need  to  know  at  company,  platoon, 
and  squad  level  are  never  made  aware  of  the  results.   Much 
emphasis  is  needed  on  informing  all  participants  down  to  the 
individual  Marine  of  the  results  of  MCCRES  [Refs.  15,37]. 
MCCRES  results  should  be  distributed  to  all  detachments  to 
ensure  they  know  how  their  performance  was  evaluated. 

Responses  to  Question  Five  indicated  that  the  orders  gener- 
ally seem  well  written  and  easy  to  interpret.   The  same  sugges- 
tion for  education  made  on  Question  Four  applies  here.   That 
is,  any  instruction  given  to  key  unit  players  will  result  in 
better  overall  understanding  of  MCCRES  and  allows  the  unit 
to  better  use  the  results. 

Several  specific  shortcomings  in  MCCRES  were  also  noted 
in  response  to  Question  Five.   Those  relating  to  aviation  were 
addressed  under  Question  Two.   One  infantry  division  has  found 
some  serious  shortcomings,  specifically  in  the  failure  to  pro- 
vide for  intelligence  gathering,  fire  support  coordination, 
and  allocation  of  tasks  from  higher  headquarters.   These  three 
areas  all  apply  to  specific  MPS ' s  that  need  expansion  or 
improvement . 
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Question  Six  by  far  evoked  the  most  emotional  and  intense 
responses.   Any  time  a  comparative  index  is  proposed  it  has 
the  potential  for  both  good  and  bad  results  depending  on  its 
use.   Of  37  individuals  interviewed,  55.4%  favored  some  sort 
of  use  of  comparative  scores.   Those  opposed  tended  to  do  so 
based  on  the  view  that  any  comparison  of  scores  provided  a 
potential  for  use  as  a  report  card  to  compare  units  and 
specifically  unit  commanders.   The  primary  objections  to  the 
use  of  comparative  scores  was  based  on  the  lack  of  overall 
standardization  within  MCCRES ,  especially  the  differences  in 
evaluators,  scenarios,  terrain,  and  weather  conditions. 

Those  who  favored  comparative  scoring  did  so  for  various 
reasons.   Four  respondents  thought  comparative  scoring  had 
value,  but  only  for  higher  headquarters,  training,  and  doc- 
trine commands.   Seven  others  thought  comparative  scoring  had 
value  if  shown  to  the  unit  in  complete  privacy.   A  final  segment 
of  ten  respondents  indicated  that  the  comparative  scoring  had 
potential  to  crea^te  more  unit  competition.   Thus,  overall,  only 
10  out  of  the  37  respondents  supported  any  sort  of  mass  publica- 
tion of  comparative  scores. 

The  results  to  Question  Seven  provided  evidence  that  feed- 
back must  be  provided  in  various  forms  throughout  the  exercise 
to  be  most  effective.   As  seen  in  response  to  Question  Three, 
64.6%  wanted  more  subjective  feedback.   In  Question  Seven 
that  was  reaffirmed.   The  respondents  said  that  by  far  the  most 
valuable  feedback  was  obtained  from  verbal,  subjective  critiques 
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given  immediately  after  an  event  has  taken  place. 
A  total  of  91.5%  of  the  respondents  indicated  a  combination 
of  debriefs  and  written  results  is  most  desirable.   The  written 
results  should  not  only  include  the  objective  numerical  re- 
sults of  "Yes/No"  markings,  but  also  explanation  of  all  the 
"No/N.A."  markings. 

Question  Eight  was  most  often  answered  with  "all  units," 
60.2%  of  the  time.   The  specific  point  made  here  was  that  the 
chain  of  command  has  an  obvious  need  to  know  the  results. 
All  units  could  benefit  from  a  generic  trend  analysis  report. 
Trend  analysis  would  provide  a  cross  pollination  of  critical 
information  throughout  the  Marine  Corps  and  assist  those  in- 
volved in  developing  training  and  doctrine.   Those  favoring 
"all  units"  or  "other  like  units"  comprised  a  total  percentage 
of  69.3%,  provided  a  favorable  response  to  the  suggestio-n  of 
receiving  a  generic  trend  analysis  at  regular  intervals  from 
the  MCCRES  Data  Bank.   The  fact  that  no  respondent  provided 
any  objection  to  this  idea  is  indicative  of  its  positive 
value. 

C .   SUMMARY 

As  can  be  seen  in  the  next  chapter  the  analysis  of  specific 
and  general  responses  to  the  eight  questions  provided  answers 
to  the  research  questions  posed  by  this  thesis.   In  addition 
a  number  of  important  unsolicited  comments,  outside  the  scope 
of  this  study,  were  also  obtained.   These  comments  are  pre- 
sented in  Appendix  B . 
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VII.   DISCUSSION  AND  CONCLUSIONS 

A.   ANSWERS  TO  PRIMARY  RESEARCH  QUESTIONS 

The  results  of  the  research  have  provided  answers  to  ques- 
tions posed  at  the  beginning  of  this  project.   The  three 
primary  research  questions  were: 

(1)  After  MCCRES ,  what  information  should  be  fed  back 
to  the  evaluated  unit  commander? 

(2)  In  what  form  should  the  evaluation  feedback  be? 

(3)  What  channel (s)  should  be  used  to  provide  the 
feedback  to  the  evaluated  unit? 

A  comprehensive  feedback  model  is  presented  which  incorporates 
the  results  for  the  three  questions. 

It  was  evident  throughout  this  study  that  the  unit  com- 
mander faces  many  constraints:   training  time,  material, 
equipment,  logistics,  and  personnel.   The  information  from 
MCCRES  should  be  provided  in  such  a  way  as  to  allow  for 
better  allocation  of  these  scarce  resources  to  the  unit. 

As  discussed  in  Chapter  II,  the  overall  goal  of  the  MCCRES 
is  not  only  to  provide  a  measure  of  unit  readiness,  but  to 
provide  effective  feedback  to  the  organization  to  enhance  the 
opportunities  to  improve  readiness  by  applying  resources  to 
areas  identified  as  deficient  by  MCCRES.   The  theories  under- 
lying management  control  and  evaluation,  as  presented  in 
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Chapter  111,    provide  a  rich  background  for  understanding  how 
to  perforin  the  MCCRES  evaluation  and  how  to  effectively  use 
the  results  of  an  evaluation.   In  both  theoretical  and  applied 
models  of  evaluation,  information  and  feedback  play  an  impor- 
tant role  in  both  performing  and  using  the  results  of  an 
evaluation.   In  general  the  m.odels  of  evaluation  have  common 
desirable  characteristics  such  as:   brevity,  clarity,  timeli- 
ness, interim  products  and  reports,  and  responsiveness  of 
feedback  [Ref .  33]  . 

Additionally/  characteristics  of  the  organizational  control 
system  [Ref.  43]  such  as  goals,  standards,  measurement  system, 
evaluation  and  reward  system  as  well  as  the  feedback  loop  play 
an  equally  important  role  in  the  evaluation  models  [Refs.  30, 
32,33,35,  and  38] . 

Based  upon  these  findings  and  the  results  of  a  field  study, 
a  model  for  MCCRES  feedback  was  developed.  The  model  is  pre- 
sented in  Figure  7.1.   Each  of  the  eight  major  elements  of  the 
model  that  affect  the  evaluated  unit  are  explained  below. 


Pre-MCCRES  Brief 


Trend  Analysis 


Follow-up  Report 


Comparative  Percentile 
Results 


Real  Time  Feedback 


Evaluator  Worksheets 


Jwo-way  Debrief 


Computerized  MCCRES  Printout 


Figure  7.1.   Model  of  MCCRES  Feedback  to  an  Evaluated  Unit 


1.  Pre-MCCRES  Briefing 

A  detailed  orientation  several  weeks  prior  to  MCCRES 
to  provide  a  firm  foundation  for  all  key  personnel.   Explana- 
tion of  appropriate  Volumes  of  MCO  3501.2.   Explanation  of 
the  details  of  each  MPS  and  procedures  for  the  evaluator 
allows  the  unit  to  maximize  the  potential  from  the  MCCRES 
feedback  [Ref.  36]. 

2.  Real  Time  Feedback 

Regular  verbal  feedback  throughout  the  exercise  by 
evaluators  at  each  level.   At  any  logical  break  in  the  scenario 
detailed  evaluations  of  the  most  recent  events  should  be  pro- 
vided by  the  evaluators.   This  provides  data  while  the  event 
is  fresh  in  everyone's  minds  [Ref.  33]. 

3.  Evaluator  Worksheets 

The  Evaluator  Worksheets  should  be  provided  directly 
to  the  evaluated  element  shortly  after  conclusion  of  the 
exercise.   This  provides  an  interim  evaluation  report  and 
allows  for  maximum  feedback  to  the  specific  element  evaluated. 
It  would  be  especially  valuable  to  subunits  and  attachments 
who  often  receive  only  minimal  feedback  under  current  proce- 
dures [Ref.  36] . 

4.  Two-way  Debrief 

At  the  end  of  exercise,  twelve  to  twenty-four  hours 
after  the  end  of  the  formal  MCCRES,  a  detailed  debrief  should 
be  held  for  evaluators  and  key  players  from  the  unit  evaluated. 
Opportunities  should  be  given  for  two-way  discussion  to  resolve 
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any  confusion  or  disagreement  particularly  regarding  subjec- 
tive areas.   The  individuals  from  the  evaluated  unit  should 
leave  with  not  only  a  clear  understanding  of  the  deficiencies, 
but  also  with  a  clear  understanding  of  what  caused  the  "No" 
evaluations  and  suggestions  for  improvement  [Ref.  33].   Since 
the  commander  of  the  unit  evaluated  is  responsible  for  the 
employment  of  the  feedback,  he  should  have  the  flexibility 
to  tailor  the  debrief  to  his  leadership  style.   The  commander 
may  desire  to  have  only  a  few  selected  evaluators  and  officers 
present  or  the  unit  commander  may  desire  all  evaluators  and 
unit  officers  to  be  present. 

5.  Computerized  MCCRES  Printout 

The  computerized  MCCRES  printout  with  percentile  scores 
and  detailed  amplifying  remarks  should  be  provided  to  the 
unit  in  a  timely  manner  [Ref.  33].   It  does  a  unit  little  good 
to  get  the  detailed  results  when  there  is  no  time  to  correct 
deficiencies.   What  is  timely  can  vary  depending  on  deployment 
schedules,  but  getting  a  copy  of  the  results  in  the  hands  of 
the  evaluated  unit  so  that  action  can  be  taken  is  essential. 
It  is  inexcusable  to  allow  staffing  to  delay  this  critical 
information.   Delays  also  put  the  unit  further  away  from  the 
MCCRES  and  it  tends  to  become  an  historical  document  vice  a 
map  for  future  training. 

6 .  Comparative  Percentile  Results 

The  data  needed  to  compare  units  within  a  division  or 
wing  is  currently  available  with  limited  research  within  the 
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division  or  wing  files.   Additionally  most  division  or  wing 
headquarters  are  well  aware  of  how  their  scores  compare  with 
other  like  units.   Usage  of  general  MCCRES  data  bank  to  provide 
comparative  scores,  medians,  and  standard  deviations  should  be 
provided  to  major  headquarters  staff,  training,  and  doctrine 
commands  for  the  limited  purpose  of  planning  future  resource 
allocation  and  to  develop  training  doctrine.   These  compara- 
tive results  should  be  limited  to  Section/MPS/Task  scores  and 
should  not  include  any  sort  of  comparative  final  score  for 
the  units  as  a  whole.   Providing  any  sort  of  overall  scores 
that  could  be  used  to  compare  individual  units  tested  could 
well  have  a  negative  impact  because  of  the  many  variances  in 
evaluation  conditions  that  exist.   It  is  further  recommended 
that  these  comparative  results  be  directed  only  at  division/ 
wing  level  or  higher  commands  to  reduce  the  chance  of  any 
type  of  unit  comparisons. 

An  example  of  comparative  percentile  results  that  could 
be  provided  to  division/wing  level  for  use  in  planning  future 
training  is  presented  in  Figure  7.2. 

PAST  SIX  MONTH  MCCRES  RESULTS 

Division/    Overall 
Wing     Marine  Corps 

Section   2.0     Operations  Performed 

MPS        2. A     Actions  by  Marines 

Task      2.A.1    Discipline 

Figure  7.2.   Sample  of  Comparative  Summary  Percentile  Results 
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7 .  Follow-up  Report 

Currently  no  follow  up  evaluation  is  made  to  con- 
firm that  the  deficiencies  have  been  corrected.   Some  type  of 
final  check  is  needed.   This  check  would  best  be  accomplished 
with  little  formality.   Since  the  unit  commander  has  the 
responsibility  for  application  of  MCCRES  feedback  to  improve 
unit  readiness,  it  is  best  left  to  the  unit  commander  to  do 
the  final  check  which  certifies  that  deficiencies  have  been 
corrected.   Resource  reallocation  from  outside  the  evaluated 
unit  likewise  should  be  certified  at  the  appropriate  level. 
The  key  issue  here  is  that  the  unit  must  be  deployed  at  the 
highest  level  of  combat  readiness  and  verification  of  correc- 
tion of  MCCRES  deficiencies  at  least  on  an  informal  basis  is 
essential.   This  follow-up  should  be  made  as  simply  as  possi- 
ble.  The  unit  commander  can  appropriately  perform  this  certi- 
fication based  on  his  own  experience  and  judgment.   A  formal 
follow-up  inspection  is  neither  necessary  nor  desirable. 

8.  Trend  Analysis 

To  gain  a  higher  level  of  across-the-board  readiness, 
a  trend  analysis  would  be  of  value  to  all  units.   The  MCCRES 
data  base  represents  a  resource  that  could  be  used  to  keep  the 
Marine  Corps  informed  of  its  overall  strengths  and  weaknesses. 
Special  emphasis  should  be  made  on  getting  these  results 
to  training  and  doctrine  organizations  throughout  the  Marine 
Corps . 
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B.   ANSWERS  TO  SECONDARY  RESEARCH  QUESTIONS 

The  results  of  the  research  provided  answers  to  the  second- 
ary research  questions  posed  at  the  beginning  of  the  project. 
The  three  secondary  research  questions  were: 

(1)  Validity  of  comparing  MCCRES  results  given  under 
differing  conditions. 

(2)  MCCRES  contribution  to  effective  training, 

(3)  Time  availability  and  follow  up  to  ensure  correction 
of  deficiencies  identified  during  MCCRES. 

A  first  response  to  the  question  of  the  validity  of  com- 
paring MCCRES  results  given  under  differing  conditions  might 
be  to  say  that  such  comparisons  can  only  be  made  if  complete 
standardization  is  achieved.   Although  complete  standardiza- 
tion for  purposes  of  evaluation  may  seem  to  be  desirable,  this 
may  not  be  feasible.   Since  various  Marine  Corps  units  (east 
coast,  west  coast,  overseas)  are  preparing  for  different  mis- 
sions, it  is  desirable  to  simulate  their  expected  combat 
employment  through  the  use  of  different  scenarios,  terrain, 
and  even  MPS's.   Additionally  the  Marine  Corps  has  other  exer- 
cises, such  as  the  Combined  Arms  Exercise  or  CAC ' s ,  which  are 
used  to  evaluate  under  standard  scenario,  terrain,  and  evalua- 
tors ,  but  they  have  a  more  limited  purpose  than  MCCRES.   Be- 
cause of  the  broader  purposes  of  MCCRES,  complete  standardization 
is  probably  not  desirable.   In  addition  it  can  be  questioned 
as  to  how  much  standardization  is  really  necessary  to  have  an 
effective  evaluation  system.   The  performance  evaluation  system 
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used  for  promotions  is  one  familiar  to  all  Marine  officers. 
It  can  be  argued  that  the  MCCRES  certainly  has  much  more 
objectivity  than  the  Performance  Evaluation  System  [Ref .  47] . 
Thus  for  its  intended  purpose,  MCCRES  appears  to  provide  a 
relatively  high  degree  of  objectivity  when  compared  with  other 
military  systems  and  the  results  are  likely  to  be  useful,  at 
least  for  generic  comparisons  and  to  identify  trends. 

In  response  to  the  question  of  MCCRES ' s  contribution  to 
effective  training,  it  is  evident  from  the  data  gathered  for 
this  thesis  that  MCCRES  has  the  potential  to,  and  currently 
does,  improve  unit  training.   Many  units  train  to  MCCRES 
standards  and  go  so  far  as  to  use  actual  MCCRES  Section/MPS/ 
Task/Requirements  to  structure  their  unit  training.   There- 
fore as  the  Individual  Training  Standards  CITS)  are  developed 
for  all  units,  it  is  essential  that  they  be  closely  compared 
with  MCCRES  standards  so  that  the  training  and  evaluation 
development  move  in  the  same  direction  [Refs.  5,28,29]. 

The  third  question,  time  availability  to  correct  MCCRES 
deficiencies,  has  previously  been  discussed.   No  doubt  more 
time  would  allow  for  more  training  in  deficient  areas.   How- 
ever, the  question  of  follow  up  is  also  critical.   At  least 
an  informal  re-validation  of  deficient  areas  is  needed  to 
ensure  the  effectiveness  of  the  MCCRES. 

C.   SUMMARY 

This  research  has  provided  detailed  answers  to  the  ques- 
tions posed  at  the  beginning  of  the  study.   The  feedback  to 


123 


the  unit  is  the  key  element  to  making  the  MCCRES  of  value  to 
the  unit  evaluated.   The  purpose  of  MCCRES  is  to  provide  timely 
and  accurate  evaluation  of  the  force  readiness.   Key  to  im- 
proving readiness  is  identification  of  strengths  and  weaknesses, 
and  providing  timely  and  thorough  feedback  to  the  evaluated 
unit  and  its  chain  of  command.   The  more  timely  and  thorough 
the  feedback,  the  higher  the  likelihood  of  improved  future 
readiness.   The  MCCRES  is  currently  viewed  as  a  sound,  valua- 
ble evaluation  system.   However,  as  this  thesis  has  demon- 
strated, the  feedback  to  the  units  can  be  improved. 

D.   STUDY  RECOMMENDATIONS 

The  feedback  model  developed  and  displayed  as  Figure  7.1 

should  be  incorporated  as  the  minimum  acceptable  procedure 

for  providing  feedback  from  any  MCCRES.   To  accomplish  this 

the  eight  steps  in  the  feedback  model  should  be  followed  during 

each  evaluation.   The  following  information  should  be  routinely 

provided  as  feedback. 

Pre-MCCRES  Briefing — to  key  MCCRES  players  from  the  evaluated 
unit 

Real  Time  Feedback — distributed  to  all  those  evaluated 

Two-way  Debrief  at  end  of  exercise — tailored  to  needs  of 
the  unit  commander  to  ensure  full  understanding  of 
evaluation  results 

Computerized  MCCRES  Printout — provided  in  a  timely  manner 
to  allow  new  resource  allocation  prior  to  deployment 

Comparative  Percentile  Results--to  help  direct  unit  train- 
ing and  for  incorporation  by  individual  training  and 
doctrine  commands 

Follow-up  Report--done  informally  by  unit  commander  to 
ensure  closure  of  feedback  loop  by  correction  of 
identified  deficiencies 

124 


Trend  Analysis--to  support  future  training  as  well  as 
changes  in  doctrine. 

Implementation  of  these  recommendations  would  suit  the 

stated  needs  of  fleet  units  as  well  as  serve  the  intended 

MCCRES  purpose  of  providing  an  evaluation  of  unit  readiness 

to  higher  headquarters. 

E.  RECOMMENDATIONS  FOR  FUTURE  RESEARCH 

The  unsolicited  comments  provided  ripe  areas  for  further 
research.  Any  item  listed  in  Appendix  B  could  be  researched 
and  could  provide  information  for  program  improvement. 

F.  SUMMARY  OF  STUDY 

The  purpose  of  this  study  was  to  determine  the  most  appro- 
priate and  effective  manner  in  which  to  provide  feedback  to 
the  unit  commander  from  a  MCCRES.   To  this  end  research  was 
conducted  into  the  areas  of  management  control  and  evaluation 
theory.   Additionally  specific  research  was  conducted  in  the 
area  of  existing  systems  which  measure  or  evaluate  readiness. 

The  study  was  approached  from  a  multi-disciplinary  back- 
ground with  a  detailed  emphasis  on  the  economic  question  of 
how  better  distribution  could  be  made  of  the  scarce  resources 
of  manpower,  equipment,  and  training  time  based  on  the  results 
of  MCCRES.   To  answer  the  questions  a  detailed  field  study  was 
conducted  in  which  interviews  were  completed  with  37  Marine 
officers  of  rank  from  Captain  to  Colonel  who  have  had  involve- 
ment with  MCCRES  as  key  billet  holders.   Additionally  much  of 
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the  original  docxunentation  that  laid  the  groundwork  for  the 
development  of  MCCRES  was  reviewed  along  with  interviews  and 
visits  with  some  of  the  original  developers  of  the  system. 

The  results  of  the  study  are  displayed  as  an  eight  step 
feedback  model  which  is  based  on  accepted  theory  in  the  fields 
of  management  control  and  evaluation  theory,  as  well  as  the 
results  of  the  field  study.   These  results  are  communicated 
to  the  reader  as  a  model  in  Figure  7.1  which  is  comprised  of 
the  following  elements: 

(1)  Pre-MCCRES  Briefing 

(2)  Real  Time  Feedback 

(3)  Evaluator  worksheets 

(4)  Two-way  Debrief  at  the  end  of  Exercise 

(5)  Computerized  MCCRES  Printout 

(6)  Comparative  Percentile  Results 

(7)  Follow-up  Report 

(8)  Trend  Analysis 

The  incorporation  of  this  model  as  standard  MCCRES 
feedback  procedure  will  significantly  enhance  the  value  of 
the  results  to  the  evaluated  unit  and  will  improve  the  under- 
standing of  the  resource  allocation  needs  at  all  levels. 
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APPENDIX  A 


SAMPLE  MARINE  CORPS  COMBAT  READINESS  EVALUATION 
SYSTEM  SOFTWARE  PRINTOUT 


2.0 

SECTION 

A) 

OPNPERFORM 

B) 

STDTACTICS 

C) 

SPECIALOPS 

D) 

EXTERSUPTD 

VOLUME  TOTAL 

2.0 
2. A 

MPS 

1)  ACTNSBYMAR 

2 )  CMD-CONTRL 

3)  FSUPTCOORD 

SECTION  TOTAL 

2.B 

MPS 


1) 

SURFASSLT 

2) 

HELIASSLT 

3) 

MVTOCONTCT 

4) 

ATTACK 

5) 

NIGHTATTCK 

6) 

DEFENSE 

7) 

RETROGRADE 

8) 

TACP 

INFANTRY 

WEIGHT 

EVAL 

100.00 

95 

45.00 

92 

18.00 

N/A 

18.00 

N/A 

100.00 

94 

INFANTRY 

OPNPERFORM 

WEIGHT 

EVAL 

100.00 

92 

58.00 

95 

50.00 

100 

100.00 

95 

STDTACTICS 

WEIGHT 

EVAL 

100.00 

N/A 

80.00 

N/A 

32.00 

100 

80.00 

96 

28.00 

68 

60.00 

93 

20.00 

N/A 

75.00 

N/A 

BY 

SECTION 

CUM  WT 

68. 

97 

31. 

03 

0. 

00 

0. 

00 

100. 

00 

BY 

MPS 

) 

SECTION  TOTAL 


45.00 


92 


CUM  WT 

33.16 
19.23 
16.58 

68.97 


CUM  WT 

0.00 
0.00 
4.97 
12.41 
4.34 
9.31 
0.00 
0.00 

31.03 
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2.C 
2.D 


SPECIALOPS   N/A 


EXTERSUPTD   N/A 


2.0 

INFANTRY 

2. A 

OPNPERFORM 

2. A 

.1 

ACTNSBYMAR 

TASK 

WEIGHT 

1) 

DISCIPLINE 

100.00 

2) 

DISPERSION 

91.00 

3) 

COVER 

100.00 

4) 

CAMOUFLAGE 

82.00 

5) 

SECURITY 

100.00 

6) 

RECONPATROL 

64.00 

7) 

COMBATPTRL 

55.00 

8) 

ELECTRONIC 

82.00 

9) 

NBC 

55.00 

10) 

CHEMICAL 

55.00 

11) 

ENEMYAIR 

82.00 

12) 

POWS 

55.00 

13) 

CASUALTIES 

45.00 

MPS  TOTAL 

100.00 

BY  TASK 


EVAL 

100 
100 
100 
100 
100 

84 
100 

43 
N/A 
100 
100 

80 

80 

92 


CUM  WT 

3.64 
3.31 
3.64 
2.98 
3.64 
2.33 
2.00 
2.98 
0.00 
2.00 
2.98 
2.00 
1.64 

33.16 
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2.0 

INFANTRY 

2. A 

OPNPERFOPvM 

2. A 

.1 

ACTNSBYMAR 

2. A 

.1.1 

DISCIPLINE 

REQUIREMENT 

WEIGHT 

1) 

SELF 

100.00 

2) 

WPNS 

80.00 

3) 

MAINT 

80.00 

4) 

FIRE 

100.00 

5) 

SUPPLY 

50.00 

6) 

COMM 

100.00 

7) 

NOISE 

90.00 

8) 

LIGHT 

90.00 

9) 

HYGENIC 

50.00 

TASK  TOTAL 

100.00 

2. A 

.1.2 

DISPERSION 

REQUIREMENT 

WEIGHT 

1) 

INDIVIDUAL 

100.00 

2) 

UNIT 

80.00 

3) 

LEADERS 

100.00 

4) 

MATERIEL 

48.00 

5) 

WEAPONS 

76.00 

TASK  TOTAL 

91.00 

BY  REQUIREMENT 


EVAL 

100 
100 
100 
100 
N/A 
100 
100 
100 
100 

100 


EVAL 

100 
100 
100 
100 
100 

100 


:ui 

4  WT 

0 

.53 

0, 

.42 

0, 

.42 

0, 

.53 

0, 

.00 

0, 

.53 

0, 

.47 

0, 

.47 

0, 

.26 

3.64 


CUM  WT 

0. 

,82 

0. 

.66 

0. 

,82 

0. 

,39 

0. 

,62 

3.31 
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APPENDIX  B 
SUMMARY  OF  UNSOLICITED  COJ^MENTS 

Because  of  the  technique  employed  of  encouraging  expansion 
and  discussion  on  each  question,  numerous  ideas  were  brought 
forth.   Although  these  were  not  specifically  evaluated,  they 
are  of  potential  importance  and  should  be  at  least  considered 
by  anyone  interested  in  MCCRES .   The  comments  identified  herein 
were  selected  based  on  their  logic  and  potential  impact  on 
MCCRES .   No  attempt  was  made  to  support  or  disprove  these 
remarks;  such  an  analysis  is,  therefore,  left  for  future 
research. 

A.   PRESENT  APPROACH  TO  MCCRES 

Few  if  any  units  fail.   To  several  respondents  the  MCCRES 
credibility  is  in  serious  doubt  because  all  units  seem  to 
pass  with  very  similar  overall  scores.   As  a  result  of  this, 
the  final  MCCRES  grade  is  not  very  meaningful,  especially  with 
no  other  grades  with  which  to  compare  the  score.   As  a  result 
of  this,  two  of  those  questioned  indicated  they  saw  no  value 
at  all  to  overall  unit  grade.   Consideration  should  be  given 
to  doing  away  with  the  unit  grade  and  just  providing  comparative- 
trend  analysis  scores  on  a  generic  basis.   This  would  do  away 
with  the  informal  comparison  of  scores  that  now  takes  place  and 
provide  better  support  of  the  intended  purpose  of  not  using  the 
MCCRES  to  compare  the  overall  readiness  of  units  evaluated. 
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B.  TWO  MCCRES'S 

The  general  feeling  among  all  the  respondents  is  that 
MCCRES  is  a  superb  opportunity  for  training  with  more  assets 
allocated  to  the  evaluated  unit  than  for  most  other  exericses. 
Some  suggest  a  MCCRES  should  be  conducted  at  both  the  beginning 
and  end  of  a  training  cycle;  to  evaluate  improvements  and  to 
measure  the  unit  against  itself.   This  would  allow  evaluators 
to  be  more  critical,  because  the  unit  would  identify  strengths 
and  weaknesses  early  and  adjust  training  objectives  accordingly. 
Although  it  is  a  costly  suggestion,  it  has  potential  to  further 
improve  readiness. 

C.  ANALYZE  INDIVIDUALS 

The  Command  needs  to  know  not^  only  strengths  and  weak- 
nesses of  unit/subunit  performance,  but  the  evaluated  unit 
commander  also  needs  an  independent  assessment  of  individual 
leadership  strengths  and  weaknesses.   Providing  MPS's  that 
require  individual  evaluation  of  leaders  at  all  levels  would 
assist  the  unit  commanders  to  reinforce  his  own  observations 
and  to  be  more  aware  of  where  he  needs  to  place  emphasis. 

D.  TACTICAL  STERILITY 

MCCRES  rewards  those  who  follow  doctrine  explicitly  and 
avoid  innovation.   In  maneuver  warfare  innovation  and  flexibility 
are  the  key  to  success.   Yet  MCCRES  measures  perfoinnance  by 
comparison  with  standard  doctrine.   Problems  here  arise  not 
out  of  the  MCCRES  procedure,  but  the  fact  that  much  of  the 
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doctrine  upon  which  MCCRES  is  based  is  rapidly  being  outdated. 
To  have  an  effective/believable  evaluation,  the  whole  cycle 
of  doctrine  and  inclusion  of  new  doctrines  in  MCCRES  must  be 
maintained.   The  general  thought  is  MCCRES  rewards  those  who 
"play  the  game"  and  may  punish  those  who  are  "ahead  of  written 
doctrine".   Some  respondents  said  sufficient  strides  have  been 
made  to  overcome  this  in  certain  suits,  but  others  said  this 
remains  a  widespread  system  problem. 

E.  LEVEL  OF  APPLICATION 

Although  Reference  1  states  that  the  Fleet  Marine  Force 
Commanding  General  is  responsible  for  application  of  MCCRES, 
the  authority  is  delegated  to  various  levels.   Some  of  the 
control  is  held  tightly  at  the  division/wing  levels;  others 
delegate  the  actual  evaluation  to  regiment/group  level.   These 
differing  procedures  greatly  change  the  standardization  of 
evaluation  and  reduce  the  pool  of  potential  evaluators . 

F.  GROUPING  AND  TIMING  FOR  EVALUATION 

Most  every  MCCRES  has  a  different  grouping  of  unit  types. 
For  instance  some  organizations  evaluate  battalion-landing 
team  units  with  the  attached  artillery,  engineers,  tanks, 
etc.,  all  evaluated  using  the  MCCRES.   Others  evaluate  only 
the  infantry  battalion;  while  some  organizations  use  the 
MCCRES  to  evaluate  artillery  as  battalions  and  others  do  not. 
The  Marine  Corps  standard  is  that  each  unit  be  evaluated  using 
MCCRES  every  two  years,  to  ensure  that  all  units  in  fact  are 
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evaluated.   A  detailed  scheduling  system  should  be  incor- 
porated to  ensure  that  no  units  are  missed  in  the  process 
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